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Preface 


Birna Arnbjérnsdottir', Branislav Bédi’, Kolbrin Fridriksdottir’, 
Holmfridur Gardarsdottir*, and Matthew James Whelpton* 


The 2022 EUROCALL conference was held in Reykjavik on 17-19 August 2022 
as a fully online event hosted by the Vigdis Finnbogadottir Institute for Foreign 
Languages, the University of Iceland, and the Arni Magnusson Institute for 
Icelandic Studies. It was preceded by an online workshop Digital Literacies in a 
Data-Driven World held on 16 August 2022 hosted by Nicky Hockly, director of 
pedagogy of The Consultants-E, an award-winning online training and development 
organisation. 


The conference theme was Intelligent CALL, granular systems and learner data. 


This theme reflects the newest developments in the field of technology for language 
learning. Subfields such as natural language processing and machine learning not 
only enable smoother spoken and written communication between human learners 
and computers, but also offer ways in which language learning can be tailored 
to the needs of individual learners. By adding components of automatic speech 
recognition, text-to-speech systems, automatic feedback mechanisms, and tracking 
systems monitoring learners’ progress and their use of tools, applications are 
becoming better targeted. All of this is used to optimise the learning experience of 
individual learners. 


The optimisation process, which is based on many different types of information 
about learners’ progress, is called granulation. Using granular systems, i.e. systems 
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equipped with mechanisms for data collection about users’ learning behaviour, is 
not only popular in assessment programmes but also in many language learning 
systems with educational purposes. From a microscopic level, granularity is 
regarded as all the different data gathered from one user. When analysing data from 
several different users of the same language learning programme, one can achieve 
a macroscopic granular view. This type of granular view allows the presentation of 
the learning behaviour of many different users in a more concise way. For instance, 
a pattern of co-occurring events on a longitudinal level can be detected. This means 
that one can learn about different points in time when individual users performed 
similar actions. 


Although there is still a long way to go before artificial intelligence, which is based 
on the combination of different technologies for language learning and user data, 
it will become an integral part of Computer-Assisted Language Learning (CALL), 
we can nonetheless say that a new technological stage of intelligent CALL has 
already commenced, and that this year’s conference could officially introduce it to 
the EUROCALL community. 


The 2022 EUROCALL conference engaged just under 280 speakers from 30 
different countries. Based on last year’s experience from our colleagues in 
Paris, we also made use this year of the Whova® platform for communication 
and sharing information, uploading pre-recorded video presentations to sustain 
the EUROCALL community, allowing for exploration of content prior to the 
conference, publishing the official conference programme, and for live streaming 
events in different virtual strands: 


¢ four keynote sessions; 
* one invited talk; 


¢ 11 group sessions in which three presenters gave a five-minute lightning 
talk with a total of 171 presentations; 


¢ six European project meetings with individual sessions in which each 
presenter gave a twenty-minute presentation; 


¢ five symposia in which three to four speakers gave short presentations in 
each symposium, totalling 21 presentations; 


* eight poster presentations in the artefact centre; 
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* six special interest group meetings; and 
¢ an editors’ workshop. 
The four outstanding keynote presentations were given by the following speakers. 


Kirsten Campbell-Howes, chief learning officer of Busuu, delivered a talk 
titled ‘Busuu: practical insights from learner data’. In her talk, she showed 
selected data of over 120 million learners, who have registered to learn a 
language on Busuu since 2008. Today, the platform generates billions of data 
points between them. In this presentation, Kirsten outlined some of the practical 
insights from this data and how these billions of data points have impacted 
Busuu’s learning design and methodology. She also described the Busuu learner 
population and some of the challenges associated with collecting and utilising 
such large datasets. 


Kolbrin Friériksd6ttir, adjunct lecturer in Icelandic as a second language 
at the University of Iceland, delivered a talk entitled ‘Framework for ways 
to promote engagement in LMOOCs’. In her presentation, she argued that 
completion rates in Massive Open Online Courses (MOOCs) are generally 
low (Jordan, 2015), which has called into question the quality of their learning 
materials and instruction, as well as learners’ goals for participation (Reich, 
2014). The need for new and engaging forms of pedagogy and design strategies 
in order to improve retention in MOOCs has thus been proposed in this context 
(Colpaert, 2014; Sokolik, 2014). Furthermore, the necessity of tutor support and 
guidance for the autonomous learner has been underlined (Ross et al., 2014). 
The main research objective in her research was to identify crucial factors that 
impact student retention and effective engagement strategies in LMOOCs. 
These factors are considered within the context of Jcelandic Online, a CALL 
programme developed at the University of Iceland. She presented her rigorous 
study (Fridriksdottir, 2018, 2021a, 2021b) where she employed mixed methods 
and used three sources of evidence: tracking data (n=43,500), survey data 
(n=400), and qualitative data (174 informants). Based on large-scale, long-term 
tracking data on a large population, the study at first provided empirical evidence 
and a detailed picture of engagement patterns and attrition across equivalent 
courses in different modes of delivery. Second, the follow-up survey in her study 
contributed to empirical evidence on language MOOC learners’ experiences of 
content-related, tutor-related, and other individual factors for engagement, and 
their influence on retention. Finally, she presented her qualitative data study, 
which offers insight into what individual learners, both completers and non- 
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completers, see as critical factors of retention or attrition in language MOOCS. 
Based on the findings of her research, she proposed a possible new framework 
for ways to promote engagement in language MOOCs. This framework includes 
several content-related strategies and tutor involvement. 


Nicky Hockly, director of pedagogy of The Consultants-E, delivered a talk entitled 
‘Intelligent CALL: the good, the bad and the ugly’. Her keynote took a bird’s 
eye view of intelligent CALL, exploring what it is, what it can do for language 
teachers, and what ramifications (both good and bad) it may have for educational 
institutions, teachers, and learners. She also took a critical look at where intelligent 
CALL is taking us, and how to ensure we get there in one piece. 


Hannes Hoégni Vilhjalmsson, professor of computer science at Reykjavik 
University, delivered a talk titled ‘More than talk: non-verbal behaviour and the 
management of co-presence in virtual language training’. In his talk, he argued that 
the typical focus of language learning materials, both traditional and interactive, 
was the mastery of spoken utterances and written sentences. While these form an 
essential core in communication, they occur within a social context. In the case of 
spoken language, he argued that the social context generally includes co-presence, 
where the bodies of those communicating occupy a common space, often defined 
by a specific social event such as a work party at a pub or a class at school. He 
mentioned that before any words are uttered, participation in conversation is almost 
always first negotiated non-verbally, through the use of position, orientation, gaze, 
posture, and gesture. While this negotiation does not rely on knowledge of a 
specific spoken language, the behaviour and its interpretation is often culturally 
dependent. He argues that it is therefore important that language learning considers 
the skills necessary to successfully manage and coordinate these situations non- 
verbally, in order to fully realise the potential of spoken proficiency. This talk 
reviewed some of the essential skills and non-verbal behaviour for managing 
co-presence and provided examples from innovative virtual social and language 
training applications. 


Our gratitude goes to all the participants, keynote speakers, invited speakers, live- 
session presenters, poster presenters, members of special interest groups, and 
workshop contributors. We are also very grateful to the rector of the University of 
Iceland for awarding an institutional sponsorship. 


Our extended gratitude goes to the EUROCALL executive committee, the scientific 
committee, and the local organising team including our colleagues and student 
helpers. 
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This volume includes 66 short papers by some of the EUROCALL 2022 presenters 
and it offers a combination of research studies and theoretical papers reflecting 
the ten subthemes of the conference. The articles are ordered alphabetically. We 
would like to thank both the authors and the reviewers for their time and effort in 
ensuring that high scientific standards have been met in delivering this volume. 
Finally, we would like to warmly thank Sylvie Thouésny and Linda Bradley for 
their immense support in the publication process and for delivering an excellent 
service in meeting the highest publication standards. 


References 


Colpaert, J. (2014). Conclusion. Reflections on present and future: towards an ontological 
approach to LMOOCs. In E. Martin-Monje & E. Barcena (Eds), Language MOOC%. 
Providing learning, transcending boundaries (pp. 161-172). De Gruyter Open. https://doi. 
org/10.2478/9783110420067.10 

Fridriksdottir, K. (2018). The impact of different modalities on student retention and overall 
engagement patterns in open online courses. Computer Assisted Language Learning, 31(1- 
2), 53-71. https://doi.org/10.1080/09588221.2017.1381129 

Fridriksdottir, K. (2021a). The effect of content-related and external factors on retention in a 
LMOOC. ReCALL, 33(2), 128-142. Special issue: Researching massive open online courses 
for language teaching and learning. https://doi.org/10.1017/S095834402 1000069 

Fridriksdottir, K. (2021b). The effect of tutor-specific and other motivational factors on student 
retention on Icelandic Online. Computer Assisted Language Learning, 34(5-6), 663-684. 
https://doi.org/10.1080/09588221.2019.1633357 

Jordan, K. (2015). Massive open online course completion rates revisited: assessment, length and 
attrition. Jnternational Review of Research in Open and Distributed Learning, 16(3), 341- 
358. https://doi.org/10.19173/irrodl.v16i3.2112 

Reich, J. (2014). MOOC completion and retention in the context of student intent. EDUCAUSE 
Review, 49(6). https://er.educause.edu/articles/2014/12/mooc-completion-and-retention-in- 
the-context-of-student-intent 

Ross, J., Sinclair, C., Knox, J., Bayne, S., & Macleod, H. (2014). Teacher experiences and 
academic identity: the missing components of MOOC pedagogy. Journal of Online Learning 
and Teaching, 10(1), 57-69. https://jolt.merlot.org/voll0no1/ross_0314.pdf 

Sokolik, M. (2014). What constitutes an effective language MOOC? In E. Martin-Monje & E. 
Barcena (Eds), Language MOOCSs. Providing learning, transcending boundaries (pp. 16- 
32). De Gruyter Open. https://doi.org/10.2478/9783110420067.2 


xix 


EU 
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Abstract. We present a study in which multimedia alphabet books were constructed 
for ten languages using the Learning And Reading Assistant (LARA) platform. We 
describe the alphabet books we built, the different design features they instantiate, 
and an initial evaluation using an anonymous online questionnaire. Links are 


provided to the books themselves, which are freely available on the web. 


Keywords: CALL, multimodality, reading, alphabets, sign language, open source. 


1. Introduction 


In order to get an initial foothold in a new language, the learner needs a basic 
understanding of three things: the sounds, the writing system, and the vocabulary. 
Although these may seem like trivial requirements, experience shows that they 
often pose serious and even insuperable obstacles. Computer Assisted Language 
Learning (CALL) methods that can address these issues are potentially very useful. 
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Suitably reconfigured for the internet age, we think there is much to recommend 
the humble alphabet book: alphabet books look simple and unthreatening, but can 
impart sophisticated linguistic information without scaring naive users. A quick 
search of the web finds many online alphabet books. There is, however, little 
systematic discussion of how they should be structured, perhaps because of an 
enduring prejudice that they are too trivial to be taken seriously. 


In this paper, we describe recent work carried out within the framework of LARA 
(Bédi et al., 2020, https://www.unige.ch/callector/lara/), an open source tool for 
creating multimedia texts, where we have used the LARA framework to create 
examples of multimodal online alphabet books for ten languages. The work has 
been carried out by several groups within the loosely-knit LARA consortium, using 
a variety of techniques. These include presentation of phoneme inventories through 
phonetic concordances or contrastive pairs; inclusion of pictures where touching 
a part of the picture plays audio; phonetically annotated words, where the student 
can spell out the word in audio form by clicking through it; and linking to online 
videos. 


2. Ten LARA alphabet books 


Table 1. Features of the alphabet books used in the study, with links to books 
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Ukrainian V 
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We built multimedia LARA alphabet books for Arabic, English, Farsi, French, 
Hebrew, Icelandic, Icelandic Sign Language, Irish, Slovak, and Ukrainian. There 
was, intentionally, no overall coordination; people were asked to develop alphabet 
books as they saw fit. In practice, people watched each other, and the later books 
often adapted features from the earlier ones. Table | summarises each alphabet 
book in terms of the following strategies. 


Audio. Clicking on a word plays audio. This was used in all languages except, for 
obvious reasons, Icelandic Sign Language. 


Images. There are pictures of objects illustrating letters. This was used for all 
languages except Ukrainian, a nonstandard alphabet book essentially designed as a 
minimalist poem making a political statement. 


Clickable images. The LARA platform supports the definition of images in 
specified areas, defined using a graphical drawing tool, which are associated with 
words (Bédi et al., 2022); hovering highlights the area, and clicking plays audio for 
the word or shows other information. This feature was used in different ways. In 
English and French, which were built around images taken from Saint-Exupéry’s 
Le petit prince, the point was to define the relevant part of the image. In Icelandic 
Sign Language, it was used to convert a poster illustrating handshapes (handshapes 
are the sign language equivalent of phonemes) into multimedia LARA form. In the 
Slovak book, which was designed for small children, it provided the basis for a 
game where each image hid the shape of the letter it illustrated. 


Video. Video was used in Icelandic Sign Language to show sign language clips, 
and for entertainment value in Hebrew. 


Examples. Four books included example sentences for letters and words. 


Phonetic text. Another feature recently included in the LARA platform is ‘phonetic 
texts’ (Bédi et al., 2022), where words are automatically divided into letter groups 
associated with phonetic values. The user clicks on letter groups rather than words. 
Clicking plays audio and also shows a phonetic concordance containing other 
examples where the same phonetic value has occurred. This feature was used for 
Arabic, English, Farsi, French, and Hebrew. For English and French, the motivation 
was to help learners understand the notoriously opaque correspondences between 
letters and sounds in these two languages. For Arabic, Farsi, and Hebrew, the point 
was to help learners link together the often quite different variant forms of letters. 
Figure | presents an illustrative screenshot from the French book. 
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Figure 1. Example of phonetic LARA text (French book): hovering over the 
letters ‘ent’ on the left highlights them, showing that they are a single 
phonetic unit; clicking shows a list of examples on the right 
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Contrasting pairs. Letters can be presented together with closely related ones. 
This was used for Irish, where one of the key issues is the palatalisation/velarisation 
contrast (Bédi et al., 2022). 


Letter variants. As noted above, Arabic and Hebrew contain variant forms of 
letters, the form used depending on the position the letter takes in the word. 


Grammar information. For English, French, Icelandic, and Irish, clicking on a 
word accesses grammatical information. For all four, the base form of the word 
is shown on the right-hand side. For Icelandic, the learner can also see a table of 
inflected forms linked from an online resource. 


3. Preliminary evaluation with questionnaire 


We posted a page” with links to the ten alphabet books and a link to an anonymous 
online questionnaire with 11 questions, and circulated it to friends, colleagues, and 
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social media, asking them to try at least one book and then answer the questionnaire. 
We received responses from 41 subjects (30 F, nine M, two O). Of these, 24 said 
they were evaluating the books from the point of view of a learner, 11 from that 
of a teacher, and six from both, while 23 subjects identified as language teachers, 
mostly (20) of English. 


The distribution of alphabet books reviewed was uneven, with 22 subjects 
reviewing English, 17 Slovak, 9 Icelandic, six Arabic and French, five Ukrainian, 
four Icelandic Sign Language, two Irish, and one Hebrew (16 subjects reviewed 
more than one book). Fortunately, English was one of the most interesting books 
in terms of including many features. On the satisfaction question (“In general, 
what was your reaction to the alphabet book you tried’), 25 responses (61%) 
were positive, with three choosing “Excellent. I will come back and use it again” 
and 24 “Interesting, I spent some time looking at it”; nine (23%) were weakly 
positive (“Kind of fun, I flipped through it a bit”). The remaining 16% were 
negative. 


On the multiple choice question “Which features do you think should be included 
in an online alphabet book for beginner learners”, the most popular answer (33; 
81%) was “audio recordings of words”. Other common answers were “Examples 
of words in a sentence” (23; 56%), “Written word for each letter”, and “Examples 
showing different forms of letters” (both 22; 54%), “Written phonetic transcripts 
of words” (20; 49%), “Written phonetic transcripts of letters” (18; 44%), “A button 
allowing learners to record their own voice” (16; 39%), and “Links to grammar 
information” (15; 37%). 


In response to the question “Based on the examples you have reviewed, would you 
be interested in using the LARA platform to construct your own alphabet book”, 
15 (37%) answered “Yes”, with most of the rest (21; 51%) answering “Don’t know/ 
haven’t thought about it”. 


4. Conclusions and further directions 


Given the fairly small sample size, our conclusions can only be tentative. Our 
impression is that both learners and teachers viewed the online alphabet books 
quite positively. Unsurprisingly, the most useful feature was perceived as being 
the ability to listen to audio, which is standard. It was also unsurprising to see 
that the majority of the subjects approved of other standard features like words 
exemplifying letters, images for the words, and example sentences. 
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Less obviously, we were interested to find that about half the subjects appeared to 
like the novel ‘phonetic text’ functionality, which was used in the Arabic, English, 
Farsi, French, and Hebrew books. More detailed investigation of user reactions to 
this feature would be a natural way to continue the study. 
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multimodal storytelling 
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Abstract. New technologies often influence the ways in which one shares linguistic 
experiences. One such technology, Augmented Reality (AR), may initiate concepts 
used in language learning through digital storytelling. This paper describes an 
exploratory research project based on Choose Your Own Adventure (CYOA) AR 
digital storytelling which investigated how Collaborative Learning (CL) and AR 
can be used to enhance language learning. Different AR activities were evaluated, 
culminating in an AR digital storytelling project. Analyses of qualitative data 
collected after each activity provided insights into important considerations when 
utilizing AR in the classroom. The findings suggest that learners found considerable 
value in AR, but also highlighted challenges in its application. CL was key to the 
success of this project. 


Keywords: augmented reality, collaborative learning, multimodal storytelling. 


1. Introduction and context 


AR can improve students’ analytical thinking and creativity by applying the 
technologies affordances, while collaborating to design and play interactive 
activities. This paper describes a CYOA digital storytelling project. It builds 
on knowledge of AR technology developed in class-based projects such as AR 
vocabulary games, AR travel fair poster presentations, and AR scavenger hunts. The 
study was conducted with Japanese freshmen English majors as part of a process- 
oriented curriculum based on the multiliteracies approach (New London Group, 
1996) with the goal of developing a sophisticated communicative competence. 
Interaction and peer-to-peer collaboration are key elements of curriculum based 
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group work and this project intended to cultivate students’ creative multimodal 
capacities. 


2: Theoretical grounding 


CL was used to both analyze and create new media forms based on the CYOA 
format. Implicit in the notion of collaboration is the idea of reciprocity and 
helping other group members to achieve a stated goal, or solve a problem, through 
cooperation rather than competition (Laal & Laal, 2012). As such, important life 
skills such as emotional intelligence, conflict resolution, teamwork, and leadership, 
as well as locating, presenting, and synthesizing information become part of the 
learning process. Also, such CL allows students to participate according to their 
relative attributes in an equitable division of labor. This characteristic of CL allowed 
students to successfully complete the CYOA project which required diverse skills 
and knowledge. 


AR immerses language learners in experiences by merging real world with virtual 
objects and images (Yang & Liao, 2014). Statti and Torres (2018) suggest positive 
learning outcomes for those who take part in AR experiential learning. As AR 
adds a digital layer to reality it provides an immersive experience which augments 
learning. This merit compliments the CYOA storytelling format which places the 
reader in the first person as an active agent making decisions in the narrative process. 
By utilizing AR these decisions can be augmented into the real world and students 
can create media which immerses the ‘reader’ into the narrative experience. AR has 
been shown to have positive effects in a range of educational contexts, including 
discovery-based learning, objects modeling, AR books, skills training, AR gaming 
(Yuen, Yaoyuneyong, & Johnson, 2011), social constructivism, and individualized 
fostering of knowledge (Zhang, Wang, & Wu, 2020). 


Creatively designing CYOA stories, therefore, seems to take advantage of CL 
and AR’s specific affordances. By requiring students to use AR as a consumer of 
a narrative, and to be active and creative designers, we required students to use 
higher-order thinking skills in their second languages. 


2 Project description 


The goal of this project was to have students think critically about the advantages 
of different textual types — written and visual, video, and AR — for storytelling, 
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thus improving their understanding of multimodal forms of expression. To achieve 
this, students collaborated to create effective AR narratives. The project was 
implemented in three stages: (1) written, (2) video, and (3) AR. Students created 
written CYOA stories and played YouTube-based CYOA videos before working 
on the AR group project. This allowed students to compare the media types and 
scaffold their knowledge as they progressed. 


Participants were first year English majors in a Japanese university (n=63) and the 
project was conducted in their second language, English. Evaluation was primarily 
based on students’ creation of the CYOA digital artifacts, while playing the other 
students’ stories was a secondary goal. After the project was completed, participants 
wrote a reflection considering the benefits and limitations of each modality and 
what they learned from the process. 


Students designed an AR CYOA story in groups of four using their second 
languages. This was a complex task requiring learners to create stories in multiple 
textual dimensions. Students considered the basic story narrative, choices and 
structure, how to tell it effectively in short video sections, and how to embed the 
story into the physical world so that the player’s physical location became part of 
the narrative. After each video, players made a decision which would direct them 
to the next location. Once found, this location revealed an AR video layer showing 
the next part of the story. Based on this structure, students collaboratively designed 
their narratives. Once completed, these were shared with the rest of the class to 
play the original AR CYOA stories. 


4. Outcomes 


The resulting CYOA stories included a detective murder mystery which directed 
the player to the scene of the crime in the physical world to look for clues, as 
well as a horror story which used enclosed physical spaces such as elevators 
to build tension. This illustrates that students thought carefully about designing 
the stories based on the strengths of AR and the use of both physical and digital 
environments. Another production consisted of an interactive, retro video game 
in which players had to find and defeat a monster that made effective use of 
sound effects. This highlighted the students’ consideration of AR modalities 
including genre and the first person camera viewpoint to immerse players within 
the narratives. Part of the appeal of AR and this project was the novelty, and 
students engaged with the challenge of creating content in an innovative medium. 
Overall, the CL aspect of this project worked especially well, as students had to 
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function effectively as teams based on their individual strengths to complete the 
task in their second languages. 


Students stated that AR was the most interesting way to tell a CYOA story. They 
especially liked the aspect of moving through the physical environment and 
collaborating with classmates to find targets and make in-game choices. They also 
felt that collaboration during the design and playing of the AR stories was enjoyable 
and beneficial to their English skills. The main constraints were frustrations with 
the AR relating to errors and not being able to go back and rewind video clips. The 
following quotes summarize the class’s general opinions on the CYOA AR project. 


“We can enjoy like a real adventure and feel reality. Otherwise, it’s little bit 
hard to make because sometimes AR has a system error. But AR activity 
is so fresh and I could enjoy it because it is my first time to make own AR 
videos”. 


“T actually tried AR, and it was so fun. Because it is like a real game and 
I can enter the game world, so it is exciting”. 


5. Limitations and future directions 


Improvements to this project would relate to enhancing the students’ awareness of 
the outcomes, specifically relating to CL life skills and metacognitive awareness. 
For instance, this might involve awareness-raising through pre- and post-task 
check sheets showing the CL skills (i.e. leadership, negotiation, and emotional 
intelligence, etc). As a language learning tool, this might be conducted in groups 
with students brainstorming language and phrases useful for each skill type. Also, a 
post-task guided group, and whole class discussion targeting effective multimodal 
communication skills would provide an excellent addition to this project. Because 
the task was relatively complex requiring different soft skills, communication 
skills, and technical skills, outcomes should be explicitly stated so that students 
become metacognitively aware of the project’s value. 


6. Conclusion 


This qualitative investigation detailed a CYOA digital storytelling project using 
AR and CL with second language learners. This approach provided a powerful 
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means for students to effectively learn soft skills in combination with multimodal 
and digital literacies. Although the technology is not perfect, students still found 
considerable value in creatively and collaboratively applying its affordances. If 
easy-to-use, reliable, and affordable AR apps become more readily available, such 
CL AR projects would be worthwhile additions to many pedagogical contexts. 
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Examining the effect of a game-informed site 
on young learners’ L2 pronunciation 


Mike Barcomb! and Walcir Cardoso? 


1. 


The use of game-informed pedagogical materials (gamification) offers cognitive, 
social, and emotional benefits in L2 learning (Reinhardt, 2019), making learning 
more enjoyable (e.g. via the inclusion of experience points) and potentially 


Abstract. This paper examines the impact that a game-informed pronunciation site 
has on the acquisition of English /r/-/l/. Twenty-three Japanese-speaking English 
learners completed a series of pronunciation activities directed at improving their 
phonological awareness and oral production of the /r/-/l/ contrast. The activities 
included game-informed tasks that rewarded learners with points, badges, and 
scavenger hunt items. For control, eight students completed the same activities 
without game-informed affordances. The study followed a mixed-methods approach 
with a pre-, post-, and delayed post-test design. Qualitative results indicate that 
learners in the game-informed group developed metaphonological awareness 
and perceived the proposed learning environment positively. For production, the 
quantitative results indicate that participants in the game-informed group improved 
their pronunciation of /r/-/l/ items. Pedagogical implications for the use of game- 


informed environments for L2 pronunciation instruction are discussed. 


Keywords:  game-informed design, gamification, L2 pronunciation, 


metaphonological awareness. 
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enabling learners to become motivated by their efforts. Of interest to this study 
is that the affordances of game-informed CALL systems have shown promise 
in L2 pronunciation instruction. Following Reinhardt’s (2019) call for game- 
informed systems to support autonomous learning and to target measurable 
outcomes, Barcomb and Cardoso (2020) developed a game-informed course 
management system that aided learners in the acquisition of L2 pronunciation. 
Young Japanese-speaking participants improved their production of /r/-/l/ and 
developed their metalinguistic awareness of the target segments. However, 
Barcomb and Cardoso’s (2020) study was short (two weeks), and it did not include 
a comparable control group. The current study addresses these limitations. 


To examine a game-informed pronunciation site designed to support autonomous 
L2 pronunciation practice (e.g. via instructional videos, listening quizzes, and 
pronunciation activities using Automated Speech Recognition — ASR), we focused 
on the acquisition of the English /r/-/l/ contrast, a feature not differentiated 
in perception or production by Japanese speakers. For control, eight students 
completed the activities in a course without game-informed elements. To this 
end, this mixed-method study addressed the following research question and sub- 
components: What effect does a game-informed course have on the acquisition 
of /r/-/l/ in terms of learners’ (RQ1) metaphonological awareness, (RQ2) oral 
production (pronunciation), and (RQ3) perceptions of the proposed game-informed 
environment? 


2. Method 


Study Design and Sample. Thirty-one young L2 Japanese learners (CEFR A-1) 
from an online school participated in the study, with 23 learners in the game- 
informed group (experiment group) and eight in the control group. All learners 
were accustomed to completing game-informed pronunciation activities as part 
of their homework; however, only the activities for the game-informed group 
included rewards. The proposed pedagogical intervention lasted six weeks. The 
control group had a different teacher, but the curriculum and lessons were otherwise 
identical. 


Procedure. Adopting Celce-Murcia, Brinton, and Goodwin’s (2010) 
communicative framework to L2 pronunciation instruction, the game- 
informed Moodle site was designed to raise awareness about the /r/-/l/ 
contrast before providing opportunities for listening discrimination and guided 
pronunciation practice. The activities included two instructional videos about 
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/r/-/l/ pronunciation, four minimal-pair listening quizzes, six ASR pronunciation 
activities with immediate feedback, and an activity where learners drew what their 
mouth looked like when pronouncing the target features. The videos emphasized 
placing the tongue on the alveolar ridge to produce /I/ and lip rounding to produce 
/r/, Each activity instantly rewarded learners with five points, which were added 
to a leaderboard. Students were encouraged to repeat activities to gain points and 
find hidden items they could add to a digital stash. See Figure | for a screenshot 
of an activity and the rewards. 


Figure 1. Features of Pronunciation Master module 


Pronunciation Master 


, Start 


Pick up! 


Instruments, Data Collection, and Analysis. A post-test questionnaire with open- 
ended responses aimed to understand if the proposed game-informed environment 
contributed to learners’ metaphonological awareness (RQ1); e.g. What do you 
know about the pronunciation of /r/ and /l/? Please explain. In line with Celce- 
Murcia et al. (2010), metaphonological awareness was operationalized as being 
able to articulate the difference/s between the pronunciation of /r/ and /I/. A pre- 
test, immediate post-test at six weeks, and delayed post-test at ten weeks with 
controlled aural elicitation tasks were administered to measure the effect that 
the proposed game-informed activities had on the participants’ production of 
/r/ and /l/ (RQ2). Participants would play a video of a teacher saying a word 
or statement before recording themselves repeating what they heard. The test 
included 32 /r/ and /1/ singletons that were evenly distributed to onset (e.g. rice) 
and coda position (e.g. tar). To understand learners’ perceptions of the proposed 
learning environment (RQ3), two written questionnaire items asked learners 
what they perceived to be the strengths and weaknesses of the game-informed 
learning environment. The questionnaire and pronunciation test were available 
on Moodle as an assignment and quiz, respectively. 
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3. Results 


Metaphonological Awareness. An analysis of the written questionnaire data 
indicates that, overall, participants developed metaphonological awareness; i.e. an 
ability to articulate their knowledge about the production of English /r/ and /I/. 
Twenty-one participants in the game-informed group were able to describe how to 
pronounce the target segments; e.g. J remember about putting my tongue behind 
my front tooth for / and not putting my tongue behind the teeth for the /r/ sound. 
Awareness was also evident in the drawing activity, which was completed by all 
the participants in the game-informed group (see Figure 2 for an example). Two 
students in the control group noted that they knew where to place their tongue to 
produce /1/, though none reported how to move their mouth to produce /r/. 


Figure 2. Sample of student drawing 


Oral Production. Results from a two-way mixed ANOVA revealed a statistically 
significant effect of time on pronunciation scores for the game-informed 
group, F(1.74, 38.25)=149.16, p<.001, partial n°=.87. Post-hoc analyses with a 
Bonferroni adjustment were conducted, revealing that the pronunciation scores 
in the game-informed group were significantly different between the pre- and 
post-test (M=5.48, SE=.38, p<.001), in addition to being significantly higher on 
the delayed post-test when compared to the pre-test (M=4.83, SE=.38, p<.001). 
There was not a statistically significant difference between pronunciation scores 
on the post-test and delayed post-test (M=.65, SE=.27, p>.05). No significant 
effects of time on the production of /r/ and /l/ scores were observed in the control 
group. Table | reports the test scores between the pre-test, post-test, and delayed 
post-test while Figure 3 provides a visual representation of pronunciation scores 
for each group. 
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Table 1. Pronunciation scores 


Pre-test Post-test Delayed Post-test 

M SD M SD M SD 
Game-informed group | 13.96 2.36 19.43 2.06 18.78 2.35 
Control group 13.13 1.73 14.38 2.77 14.00 3.16 


Figure 3. Composite oral production scores 


Pronunciation Scores 


20.00 Group 


=~ Control 
—— Game-Informed 


18.00 


16.00 


14.00 — 


Time 


Composite /r/-/l/ score 


Users’ Perceptions. An analysis of the written questionnaire data indicates 
that, overall, participants in the game-informed group perceived the experience 
positively. Eighteen students enjoyed competition (e.g. J like competing and 
collecting experience points). Twenty-two students reported that the elements 
motivated them to review materials (e.g. Stash items motivated me to study more 
because I went back to study to find the stash items). These responses also shed light 
on why the pronunciation results may differ between the groups, as the experiment 
group completed an average of 17.13 activities while the control group completed 
an average of only 6.50 activities. 


4. Discussion and conclusions 


This study provides evidence for the potential of game-informed pedagogy in L2 
pronunciation instruction. While the game-informed group showed a statistically 
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significant improvement of /r/-/l/ production on the two post-tests, the control 
group did not show any improvements. This finding is likely due to the game- 
informed elements, which encouraged participants in the experimental group to use 
the materials repeatedly (almost three times more frequently than participants in 
the control group). These results also align with the predictions of Celce-Murcia et 
al.’s (2010) framework, as an instructional focus on metaphonological awareness 
and controlled practice resulted in production accuracy. These types of practices 
increase learners’ ability to process the target feature in short-term memory (Celce- 
Murcia et al., 2010), subsequently transferring that knowledge to oral production, 
as observed in this study. 


Due to working with students in a pre-existing online school, the sample size for 
this study was relatively small and may affect the generalizability of the findings. 
Despite this limitation, the participants were accustomed to using game-informed 
materials, thus reducing the novelty effect seen with game-informed materials. 
This study also embodied a praxis perspective to game-informed second language 
teaching and learning, which assumes that theory, research, and practice contribute 
equally to developing and testing resources in a shared gameful experience 
between teachers and students (Reinhardt, 2019). Future studies should examine 
the impact of game-informed pronunciation practice on the development of 
different phonological features of English (e.g. rhythm, stress) and implement more 
conversational aspects of pronunciation practice such as guided and communicative 
practice. 
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Abstract. We argue that museums presenting exhibits of ancient texts may be 
able to benefit from accompanying them with annotated electronic versions. We 
present a short study using two sample annotated texts built using the Learning And 
Reading Assistant (LARA) platform for a fragment of an Old Norse manuscript and 
an inscription in Ancient Egyptian hieroglyphs respectively. The results of an online 
questionnaire suggested that people already interested in ancient languages would 
view the idea very positively. 


Keywords: multimodality, museums, manuscripts, inscriptions. 


1. Introduction 


Most people interested in archaic languages have probably had the experience 
of standing in a museum, examining a manuscript, inscription, or tablet, and 
wondering what the signs written on them mean. Some exhibits may have a 
plaque next to the original source with some explanatory text. But even in the 
best case, when a full translation is provided, what is the connection between the 
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signs, letters, and the claimed meaning? How much of the meaning is present 
in the translated text provided, and how much is guesswork? Here, we present 
an initial experiment, where we have used the open-source LARA (Bédi et al., 
2020b — https://www. unige.ch/callector/lara/) to create a basic prototype of what 
a digitally enhanced museum exhibit of this kind might use to inform and educate 
their visitors. LARA supports creating multimedia resources with translations into 
various languages depending on the creator’s purpose. Specifically, we produced 
multimedia versions of two sample documents, an extract from an Old Norse 
manuscript and an inscription in Ancient Egyptian hieroglyphics. Each multimedia 
text is presented in two different views. The Old Norse text offers a ‘manuscript’ 
view and a ‘printed text’ view. The hieroglyphics text offers a ‘sign’ view, which 
displays information about the individual glyphs, and a ‘words’ view, which groups 
them into words and sentences. Hovering over a part of the text that is associated 
with an annotation outlines the symbol or image and shows a popup gloss; clicking 
plays audio and displays other information. In sections below, the methodology, 
evaluation of results, and comments will be discussed. 


2. Two annotated texts in ancient languages 


We present the two annotated texts used. Both can be accessed from a summary 
page® which also contains a link to the questionnaire described in the following 
section. 


2.1. A fragment of an Old Norse manuscript 


The first document (Figure 1) is directly available online’ and consists of three 
verses taken from the traditional Old Norse epic VGluspa (The Seeress’s Prophecy), 
among other things famous as Tolkien’s favourite poem. The LARA document, 
illustrated in Figure |, has been adapted from a full LARA version of Véluspa 
developed under a previous project (Bédi et al., 2020a). Most importantly, the 
manuscript view was added using the methods described in Bédi et al. (2022): as 
explained there, this involved using a graphical tool to draw the area in the image 
associated with each text word. Every verse includes audio and a parallel line-by- 
line translation. The user can hover over or click on words in either view. Hovering 
over a word in the manuscript view outlines it and shows a popup with a plain text 


8. https://www.issco.unige.ch/en/research/projects/callector/manuscripts_and_inscriptions.html 
9. https://www.issco.unige.ch/en/research/projects/callector/v%C3%B6lusp%C3%A |_manuscript_v3vocabpages/_hyperlinked_ 
text_.html 
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transcription and a gloss translated into English. Clicking plays audio and shows 
other information. 


The overall intent is that the reader should both be able to understand the verses and 
also to have some chance of appreciating them as poetry. It is obviously difficult 
to quantify the second goal, but our own experiences and informal exchanges with 
other people who have looked at the material lead us to believe that the dramatic 
quality of the audio recording is key. An earlier version with a lower-quality voice 
was unconvincing: the one posted, which was recorded by a professional-quality 
voice talent, has frequently been described as ‘beautiful’ or ‘moving’ even by 
people who have no previous knowledge of Old Norse. 


Figure 1. A verse from V6éluspd. Each verse is presented both in the original 


1280 manuscript form and also in a print version. Here, the reader has 
clicked on salar (‘earth’s’) in the manuscript version (highlighted in a 
green frame). The lower pane shows that the word in red also occurs in 
another verse. 


Voluspa 


Here, you can see three verses from Véluspd, Tolkien's favourite Old Norse poem. Each verse is 
presented both in the original 1280 manuscript form and also in a print version. Hover over words in 
either version to see translations, click on them to hear the word spoken. 


Verse 4. The creation of Middle Earth. 


salar; earth's 


salr 


Grammar information 


Lexicon poeticum entry 


< Aér Burs synir bj6dum um yppdu, peir er Midgard mzran sk6pu; s6l skein sunnan 4 salar steina, b4 var 
grund gréin grenum lauki. 


<= S6l varp sunnan, sinni mana, hendi inni hegri um himinjédyr; sol bad né vissi hvar hon sali atti, stjérnur 
pad né vissu hvar ber stadi attu, mani pad né vissi hvad hann megins atti. 


Notes 


Frequency index 


Alphabetical index 
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2.2. Aninscription in Ancient Egyptian hieroglyphs 


The second document (Figure 2) is available online’? and presents an Ancient 
Egyptian inscription taken from a popular introductory text on hieroglyphs (Collier 
& Manley, 1998). Again, the methods from Bédi et al. (2022) have been used to 
associate appropriate areas in the image with words and individual glyphs. In this 
example, the central goal is to make the apparently indecipherable hieroglyphs 
comprehensible, so that the visitor can experience the unfamiliar and enjoyable 
sensation of reading and understanding them. In contrast to the Old Norse example, 
the audio is of less importance and is primarily decorative. The LARA multimedia 
version presents the Ancient Egyptian inscriptions in two versions: the first version, 
left, shows it divided up into words, and the second version (right) shows it divided 
up into individual hieroglyphs. In both cases, hovering over something shows a 
popup with a gloss, and clicking shows more information and plays audio recorded 
by a native Egyptian male voice talent. 


Figure 2. This inscription (highlighted in green frame) says, roughly: ‘Ox-leg 
for the ka (soul) of Senbi, the virtuous’. Here, the reader has clicked 
on the highlighted word snbi (the name ‘Senbi’). This brings up the 
explanatory linearised text on the right and plays an audio version. 


Version divided into words 
snbi 
CY | ae | Bo 
e a — 2 ponsoon 
U re leer? (ee 
Frequency index 


10. https://www.issco.unige.ch/en/research/projects/callector/hieroglyphics lavocabpages/_hyperlinked_text_.html 
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3. Preliminary evaluation with questionnaire 


We circulated a link to the summary page hosted on the University of Geneva’s 
server, encouraging people to look at the two examples described in the preceding 
section and fill in a short anonymous questionnaire with nine questions created 
in Google Forms, which was distributed on social network sites for learning 
languages. We received responses from 24 subjects (12 female, 11 male, one 
other). In response to the first question, Generally, are you interested in texts 
written in ancient languages?, 18 (75%) replied “Yes” and four (17%) replied “A 
bit”. In response to the second question, When you visit museums, do you like to 
look at texts in ancient languages?, we received 16 answers of “Yes” and eight of 
“A bit”. In response to the third question, What was your reaction to the examples 
of the multimodal ancient texts here?, 17 subjects (71%) gave the most positive 
answer (“Very interesting, I spent some time looking at them’’) and the other seven 
(29%) gave the second most positive answer (“Kind of fun, I flipped through them 
a bit”). In response to the fourth question, Would you be interested in accessing 
similar multimodal texts in ancient languages on a museum website?, 18 (75%) 
selected “Very interested” and four (17%) selected “A bit interested”. In the fifth 
question, where a list of statements was presented and subjects were asked to 
tick all the ones they agreed with, 20 (83%) ticked “Multimodal texts like these 
would help me understand text exhibitions at museums” and nine (38%) ticked 
“T like to read about museum exhibitions before or after I visit, I might use them 
then”. The sixth question invited subjects to give free-form comments about 
how the annotated texts could be improved. Unsurprisingly, given that much less 
work has been invested in it and it is still rather rough, most of the criticism was 
directed towards the hieroglyphics example; it was also apparent that subjects 
would have liked the material to be easily available on mobile devices. However, 
in the seventh question, which asked for general free-form comments, all replies 
were extremely positive. 


4. Conclusions and further directions 


The large number of positive responses to the first two questions on the questionnaire 
suggest a self-selection effect: people interested in ancient languages are evidently 
overrepresented in this population. Also, the sample is fairly small. With these 
caveats, our impression is that people who like ancient languages would welcome 
the introduction of this kind of technology in relevant museums, and would 
perceive it as substantially enhancing the museum experience. Encouraged by this, 
we have begun to discuss the idea with people active in the museum world. Their 
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initial response has also been positive: in particular, they seem attracted to the idea 
of making text exhibits more open, technologically enhanced, and accessible. 
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Abstract. This paper introduces FineTune, a mobile application currently in 
development that features a gamified version of High Variability Phonetic Training 
(HVPT). The aims of this paper are twofold: (1) to outline the theoretical motivation 
for the app’s features, which includes a discussion of HVPT, gamification, and 
Self-Determination Theory (SDT), and (2) to describe how FineTune functions as a 
mobile application, considering the theories that informed its design. 


Keywords: gamification, mobile-assisted language learning, high variability 


phonetic training, pronunciation. 


1. Introduction 


HVPT, as stated in Black-Rotchin’s (2022, p. iii), is an established method of 
improving the learner’s perception and potential production of second/foreign 
language (L2) segments, which has received considerable attention by the 
research community (Barriuso & Hayes-Harb, 2018). However, this technique 
is unknown to most teachers and is rarely translated into usable pedagogical 
tools (Thomson, 2018), despite the known issue that many teachers feel uncertain 
about how to effectively teach pronunciation (Foote, Holtby, & Derwing, 2012). 
Thomson (2018) notes that one reason for this disconnect may be that in its 
current lab-based form, HVPT is neither accessible nor engaging to L2 learners. 
In order to mitigate both issues, he suggests implementing this training into a 
mobile application and adding game elements to the existing HVPT paradigm. 
He posits that such changes may help in bridging the gap between pronunciation 
research and pedagogical practices. 
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In response to these suggestions, this paper introduces the blueprint for FineTune, a 
mobile application that features a gamified version of HVPT to help learners hone 
their perception of L2 English sound systems. The perceptual training in Fine Tune 
involves learners progressing through three levels of gamified forced-choice 
identification tasks whereby learners select the correct target sounds according to 
the stimuli heard (Figure | illustrates the training of /e/, the vowel in ‘bet’). They 
also have the options of testing their production skills through a speech recognition 
feature and joining a learning team to connect with other learners. 


Figure |. The interface of FineTune 


ep 
a %) @ 


The design of FineTune is informed by selected variables of past HVPT studies, 
as well as Stockwell and Hubbard’s (2013) ten principles for developing MALL 
tools. The game elements of the app were chosen based on van Roy and Zaman’s 
(2017) nine gamification heuristics for educational contexts, which itself is 
informed by Deci and Ryan’s (2000) SDT. These constructs will be explored in 
the following section. 


2. FineTune: theoretical motivation 


2.1. HVPT 


HVPT is a form of pronunciation training that constitutes the basis for FineTune. 
HVPT isa method of improving learners’ perception (and consequently production) 
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of speech sounds by exposing learners to multiple voices producing the same target 
sound/s. HVPT has been shown to produce perception gains of consonants (Cebrian 
& Carlet, 2014), vowels (Lambacher et al., 2005), and tones (Perrachione, Lee, 
Ha, & Wong, 2018) with learners of varying L1s. Within FineTune, HVPT tenets 
appear in the variability of speakers (e.g. different ages, gender) and phonetic 
environments of the target sounds (e.g. in word-initial and word-final positions), as 
well as in the forced-choice identification tasks that are accompanied by immediate 
feedback. 


2.2. Gamification 


Gamification is the process of adding game elements to non-game contexts, with 
the aim of fostering gameful and playful emotions that increase engagement 
(Hernandez-Gonzalez, 2021). SDT explains why certain game elements succeed in 
cultivating this engagement: they satisfy the basic human need to feel autonomous, 
competent, and connected to others (Deci & Ryan, 2000). In FineTune, game 
elements are selected based on their empirically-demonstrated ability to satisfy 
at least one of these three needs. Game design is also informed by van Roy and 
Zaman’s (2017) gamification heuristics, which outlines how game design (not just 
the selection of game elements) can be harmonious with SDT. 


2.3. | Mobile-Assisted Language Learning (MALL) 


MALL allows for student-centered learning that offers an unprecedented degree of 
customization in terms of where and how the learning takes place. The limitations 
and affordances of the medium are taken into account in the design of FineTune 
by following Stockwell and Hubbard’s (2013) principles for developing MALL 
tools. For instance, multi-tasking is limited, tasks are kept short and manageable, 
and users’ autonomy and individual differences are considered in the design of 
activities. 


3. FineTune description 


In FineTune, learners practice perceiving and producing sounds using a set 
of gamified forced-choice identification tasks that are broken down into three 
scaffolded levels. Before a learner begins training, they are prompted to create a 
profile that suggests sounds to target based on their linguistic repertoire. They are 
also prompted to create a specific and measurable goal to guide their training (e.g. 
to learn English interdentals by a given date). Learners are also given the choice 
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of joining a learning team of others who are practicing the same sounds. After 
creating a profile, they can start progressing through the levels. Level | involves 
identifying the target sound(s) in non-words, Level 2 in real words, and Level 3 in 
carrier sentences. In the training, users collect points toward badges that represent 
mastery of a sound. Once they have completed all levels with a perfect score, they 
are awarded the badge, which is added to their profile. 


FineTune attempts to accommodate the three needs recommended by SDT: user 
autonomy, competence, and connectedness. User autonomy is present in FineTune 
in many ways: users can choose which sounds to practice, they can decide to join a 
team, and they can opt out of certain game elements, such as time limits. They set 
learning objectives for themselves and have the ability to share and gain knowledge 
from other users. In addition, many of the game elements in Fine Tune instill feelings 
of increasing competence. Levels are scaffolded based on difficulty, points and 
badges act as signifiers of skill, forums allow for the exchange of information, and 
progress bars on users’ profiles indicate their proximity to achieving their goals. 
Relatedness is accounted for in FineTune through the learning team, whereby users 
can feel that they are not alone in their endeavor; should they wish, they can solicit 
the help of another learner, or offer assistance themselves. 


Finally, FineTune incorporates other game elements such as the use of musical 
motifs and aesthetic design, time limits, and the tactile nature of the perceptual 
training. 


4. Concluding remarks 


The main goals of this study were to motivate and map out the blueprint for the 
mobile application FineTune, a mobile application developed by the authors that 
features a gamified version of HVPT. This conceptual study represents Stage 1 of 4 
in Cardoso’s (2022) chronological description of how computer assisted language 
learning research is conducted, whereby the motivation and creation for the app is 
described prior to investigating its pedagogical affordances, user attitudes, and its 
pedagogical efficacy in the learning of L2 pronunciation. Hopefully, by chronicling 
the development of a MALL tool from inception to creation, more such studies 
will explore the tangible bridging of research findings and pedagogical tools for 
educators and learners to use. 


The next step in development is to build a prototype of the application, which 
will likely involve modifications to the current design. These changes will be 
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documented as part of the process. Once a prototype has been completed, the 
pedagogical efficacy of the tool will be assessed, as well as user attitudes. One 
possible way of carrying out this assessment is by using the technology acceptance 
model (Davis, 1989) which examines users’ perceptions of the usefulness and 
useability of a tool. 
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Abstract. This paper examines learners’ perceptions of using Moodle books in the 
online course module Contemporary society: conflicts and consensus at Casa Grande 
University, Ecuador. It describes how learners perceived the use of Moodle books in 
48-hour English content based courses. Courses were designed for final year students 
with a B2 English proficiency level according to the Common European Framework 
of Foreign languages (CEFR). The learners used Moodle books to access content 
organized and programmed in six thematic units according to the course learning 
objectives. Learners used content to access reading material and complete tasks 
planned synchronously and asynchronously. Learners responded to a post course 
survey in courses that ran from 2020 to 2021. The purpose of the survey was to 
know how learners perceived the organization, functionality, and effectiveness of 
Moodle books when accessing hyper-texted content, multimedia resources, and 
digital tools for content management, communication, and interaction in online 


courses for English for Specific Purposes (ESP). 


Keywords: Moodle VLE/CMS, online courses, learning platform. 


1. Introduction 


Over the past 20 years, the use of online learning platforms such as Moodle, it's 
learning and blackboard has become commonplace in universities all over the 
world. It is possible to see these platforms as Virtual Learning Environments 
(VLEs) where students engage in learning activities relating to their course module 
and as purely administrative tools to communicate with course participants, update 
grades, or track progress (Course Management Systems — CMSs), or as a mixture 
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of both conceptualizations. An early introduction to VLE/CMSs and a discussion of 
the institutional choices involved in the adoption of online learning platforms can 
be found in Weller (2007). With the recent pandemic increasingly forcing higher 
education institutions to adopt online or blended solutions, the place of online 
learning platforms at university level seems assured, at least for the time being. 


The rise of the VLE/CMS phenomenon has put increasing pressure on teaching 
staff to become ‘learning designers’ who need to carefully consider the structure, 
clarity, and ease of navigation of the materials they put on the platforms. An 
additional requirement is that learners should be able to access VLE/CMSs through 
the interfaces of mobile phones and devices as well as stationary computers. 


One consequence of these concerns is that many Moodle sites may not be 
adequately designed, with long sequences of incoherent and haphazardly 
arranged materials, forums, and assignment holders. At this point it seems fair 
to say that technology is not fully serving its purpose when/if content is not 
properly categorized and organized. In this respect, Egbert (2005) points out that 
“technology accomplishes learning goals with less time and work for teachers 
and learners” (p. 12) in guidelines for using educational technology in language 
classrooms. This is obviously not the case when/if Moodle activities are time 
consuming and disconnected. 


Another consequence is that Moodle sites may not be intuitive enough. In other 
words learners may not access and interact easily if content is not arranged and 
presented properly. Learners are Moodle independent site users and make their own 
choices in the VLE. Richards and Lockhart (1996) describe some characteristics 
of language learners when he states that “learners, too, bring to learning their 
own beliefs, goals, attitudes, and decisions, which in turn influence how they 
approach learning” (p. 52). They need a clear path to manage content and navigate. 
Therefore, content design and presentation play an important role in facilitating 
content navigation and interaction. 


In Moodle, platform developers have attempted to counteract these problems 
through the use of so-called ‘books’ which enable teaching staff to compress and 
structure large amounts of text and graphics in the form of numbered chapters and 
subchapters. Referring to the scientific basis of learning published in the volume 
How people learn, Bransford, Brown, and Cooking (2003) state that “[I]earning and 
understanding can be facilitated in learners by emphasizing organized, coherent 
bodies of knowledge” (p. 239). This volume reports on scientific research on the 
new science of learning, providing information about: 
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* memory and structure of knowledge; 

¢ problem solving and reasoning; 

¢ the early foundations of learning; and 

¢ regulatory processes that govern learning, including metacognition 
(Bransford et al., 2003, p. 14). 


The purpose of this study is to describe learners’ perceptions of the use of Moodle 
books in ESP courses. The study is within the context of higher education in South 
America’, a continent which in some quarters have been seen to be lagging in terms 
of digitalization. It looks at how 49 learners accessed Moodle books through the 
course menu. The menu was structured in six units and unit tasks. It additionally 
included menu options such as a general bulletin board used to coordinate 
activities, tests, and events, a ‘coffee lounge’ for social interaction, and a wiki 
to maximize research among participants. Learners used a Moodle book in every 
unit to access topics and subtopics organized in chapters and subchapters. They 
read hyperlinked material and used digital resources to complete tasks sequenced 
numerically through the main menu. Moodle books included visual aids, videos, 
and presentations to support and develop content. Students’ perceptions were 
collected in a Google survey form at the end of the course. 


2. Method 


The students, aged 20 to 22, took the course in the third year of studies. The 
students were all Spanish native speakers but they used English in ESP classes 
to communicate, interact, and complete their tasks. All of them had a B2 English 
proficiency level which is an academic requirement to take advanced English 
content subjects. 


The data source was a post-class survey. Forty-nine students answered the survey 
in Google Forms at the end of the courses from 2020 to 2021.The aim of the 
survey was to gather information about students’ perceptions and feedback of the 
organization, functionality, and effectiveness of Moodle books. Students responded 
to seven questions. All questions focused on students’ opinions about the use of 
Moodle books in their courses. The first six questions included different items of 
choices related to the use of Moodle books in online courses. Items were ranked on 
a Likert grading scale according to students’ individual choices. Question 7 was an 
open question requiring students’ additional comments. 
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3. Results and discussion 


The results from the survey are presented below in the form of a fairly self- 
explanatory histogram of the responses from the students (Figure 1). 


Figure 1. 


Learner responses to the survey questions, rated a Likert scale of one 


(no agreement) to seven (complete agreement) 


1. Moodle books helped me to keep track of all the texts and documents relevant to 
the course 
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5. Moodle books enable me to easily access multimedia (audio and video ) 
Information related to the reading texts, 


2. Moodie books help course participants to navigate easily in the Moodle site 
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4. Using Moodle books it is easy to see the relationship between reading texts and 
the assignments we need to do for the course 


49 responses, 
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6. Moodle books help me to cope effectively with information overioad which would 
‘otherwise be the result of having to scroll through longer pages of separate files 
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As can be seen from the above histogram plots, responses from the students were in 
general very positive to the use of Moodle books in the structuring and presentation 
of information on the Moodle course site pages. This positive perception related 
to the ease of navigation on the site both from stationary computers and mobile 
devices such as cellphones and tablets as well as establishing a clear link from 
the course reading comprehension texts and the assignments required for the 
assessment of the course. The responses also pointed toward easing work loads 
and increasing learning effectiveness which are important factors for this target 
group of students, many of whom are studying alongside full- and part-time 
employment. 


Twenty seven out of 49 students posted survey comments that mostly highlighted 
aspects of Moodle books as shown in a representative selection of students’ 
answers. 


33 


Maria del Carmen Bolofa and Christopher Allen 


“The Moodle books are of great use and make work easier”. 


“They are great for an introduction, it helps keep track of what we’ve 
seen”. 


“The information in the books were more than enough to understand and 
complete the assignments, they were a great addition”. 


“T consider Moodle books to be a summary of a specific topic. Being that 
they guide me on a topic”. 


4. Conclusions 


The results are perhaps not particularly surprising in the sense that Moodle 
books are seen to bring a number of positive outcomes in this wholly online 
context. The main findings seem to indicate at least for this small sample that 
course participants are very much aware of appearance and structuring of 
information on the course pages and that these design aspects may be important 
factors in their successful completion of the course alongside employment 
responsibilities. 


There is a clear message to university academics designing courses either wholly 
online or in blended mode that the packaging and structuring of information and 
the graphic appearance of page content are very important to ensure learning 
progress and course completion. University teachers are of course content 
experts in the subjects they are teaching, but they are unlikely to possess the 
skills of a graphic designer to aid them in the process of course design. Moodle 
books however offer teaching staff the possibility to produce simple and easily 
navigable course sites where information is packaged in an readily accessible 
format. This is certainly one area in which institutional educational technologists 
can assist academic teachers in developing their course design and graphic 
presentation skills. 


Students perceive positive effects in content access, management, and interaction 
when they use Moodle books. This research has shown the importance of 
incorporating students’ views in the design and development of ESP courses 
where Moodle books’ organization, effectiveness, and functionality can maximize 
students’ learning results. 
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Abstract. This essay illustrates how an online language education course, implemented 
within an online language teacher education programme offered at master’s level 
at an Italian university, was designed to foster pre-service teachers’ active learning 
through digitally-enhanced activities. The course was developed within a community 
of inquiry framework and from a socio-constructivist perspective. The online course 
adopted a flipped learning approach. Pre-service teachers carried out asynchronous 
digitally-enhanced activities individually and collaboratively before class and 
synchronous technology-enhanced collaborative activities during live classes. Pre- 
service teachers thus engaged in active learning throughout the course. 


Keywords: online learning, language teacher education, active learning, educational 
technology. 


1. Introduction 


The Covid-19 disruption has fostered the digitisation of education extensively. In this 
new context, teacher education programmes have had to re-imagine their learning 
spaces and practices to cater to teachers’ pedagogical needs in online learning 
environments (Assuncao & Swennen, 2020). Online course design has thus become 
pivotal in higher education worldwide. In this respect, the present essay aims to 
illustrate how an online language education course was designed to foster pre-service 
language teachers’ active learning through synchronous and asynchronous digitally- 
enhanced activities. The course was implemented within an online language teacher 
education programme, offered at master’s level at an Italian university, targeted at 
pre-service teachers specialising in teaching Italian as a second language. 
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2. Online language education course design 


In the online language education course, pre-service teachers learned how to 
plan and create digitally-enhanced language teaching units. The online language 
education course was designed within a community of inquiry framework 
(Garrison, Anderson, & Archer, 2000, 2001; Vaughan, Cleveland-Innes, & 
Garrison, 2013) and from a socio-constructivist perspective, enhancing knowledge 
co-development (Lantolf, Poehner, & Thorne, 2020; Lantolf & Thorne, 2006; 
Vygotsky, 1978). In keeping with the community of inquiry model, the course 
instructor created practices and activities targeted at fostering social, teaching, 
and cognitive presence (Garrison et al., 2000, 2001; Vaughan et al., 2013). The 
course was developed using backward design; the instructor thus devised the 
course learning outcomes first, the assessment criteria second, and the strategies 
and activities targeted at achieving the course objectives afterwards (McTighe & 
Wiggins, 2012). 


The online language education course was designed to foster pre-service teachers’ 
active learning with the support of digital and educational technologies. In active 
learning, students engage in knowledge construction using higher-order thinking 
skills (Bonwell & Eison, 1991; Borte, Nesje, & Lillejord, 2020; Lee, Morrone, 
& Siering, 2018; Prince, 2004; Stefanou et al., 2013). In this light, the instructor 
adopted a flipped learning approach to design digitally-enhanced pedagogical 
practices promoting critical thinking (Bergmann & Sams, 2012, 2014; Marshall & 
Kostka, 2020; Marshall & Rodriguez-Buitrago, 2017). Pre-class activities required 
pre-service language teachers to engage with subject-specific content and peers to 
various degrees. For example, before Zoom-based’ classes, pre-service teachers 
watched videos, previously recorded by the instructor, and carried out activities, 
focusing on the video content, using various digital technologies, such as avatar 
maker tools, mind mapping, peer assessment tools, Al-driven technologies, and 
digital social educational platforms suitable for collaborative annotation of texts 
and videos. In this respect, a digital social educational platform used, Perusall’, 
provided the instructor with reports, based on data analytics, highlighting the most 
challenging topics for students; the instructor could also access reports on individual 
students’ degree of engagement with the assigned materials. These data enabled the 
instructor to design synchronous activities catering to pre-service teachers’ needs. 
During synchronous classes, activities were highly chunked in line with digital 
pedagogy (Bates, 2019). In live classes, pre-service teachers engaged with tasks 
fostering higher-level thinking skills and a deeper conceptual understanding of the 
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subject-specific content studied during pre-class asynchronous activities (Brinks 
Lockwood, 2014, 2018). Synchronous classes started with ice breakers, pivotal to 
promoting social presence in online spaces; the instructor implemented ice breakers 
with the support of digital noticeboards or digital pedagogical platforms suited to 
promoting remote collaboration. During Zoom-based classes, pre-service teachers 
engaged in collaborative learning through peer instruction (Mazur, 1997), SOFLA 
(Synchronous Online Flipped Learning Approach — Marshall & Kostka, 2020), and 
liberating structures activities (Lipmanowicz & McCandless, 2014). In synchronous 
classes, pre-service teachers thus engaged in active learning through collaborative 
activities implemented with the support of collaborative annotation spaces, digital 
noticeboards, and digital educational platforms suitable for remote collaboration. 
Digital and educational technologies, along with digital educational platforms, were 
instrumental in developing cognitive and teaching presence in the course. 


Pre-service teachers developed a digitally-enhanced language teaching unit 
collaboratively during the course. Each week, as home assignments, pre-service 
teachers devised a section of the digitally-enhanced teaching unit in small 
groups. Before synchronous classes, the instructor assessed the artefacts created 
collaboratively and provided each group with video recorded feedback, also suited 
to fostering social presence. Pre-service teachers developed a digitally-enhanced 
teaching unit collaboratively as their final assignment. Before submitting the units 
for assessment, each group received video recorded feedback from the instructor; 
pre-service teachers could use the feedback to make some changes to the units 
before submission. 


At the end of each live class, pre-service teachers filled in an online self-evaluation 
questionnaire. The instructor used students’ feedback, gathered through the online 
self-evaluation questionnaires, to monitor the course and make some changes if 
necessary. On the last day of class, the instructor introduced pre-service teachers 
to the theoretical framework, namely the community of inquiry model, adopted to 
design the course. 


3. Conclusion 


The fast digitisation process occurring in higher education has enabled course 
instructors to re-image their learning practices and spaces. Re-thinking online 
learning environments has provided language education practitioners with the 
opportunity to design online learning pathways fostering students’ active learning 
and engagement to a very high degree. 
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This essay has illustrated how an instructor designed an online language education 
course fostering pre-service teachers’ active learning and engagement through a 
flipped learning approach. The asynchronous and synchronous digitally-enhanced 
practices developed were targeted at developing pre-service teachers’ higher-level 
thinking skills and deeper conceptual understanding of subject-specific content. 
Knowledge co-construction was scaffolded through the use of various digital and 
educational technologies and platforms instrumental in fostering collaborative 
learning and artefact creation. 
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1. 


The trilateral partnership underpinning this project was initiated in March 2020 
when Europe went into its first Covid lockdown. Subsequently, a six-week pilot 
project (virtual-only), supported by Unicollaboration, ran from November until 
December 2020, with 36 students from Malaga, Bremerhaven, and Letterkenny 
participating. Building on this, the current project adds a physical element to create 
a BIP. This article will begin with a brief discussion of the concept of blended 


Abstract. Within the parameters of the new Erasmus programme, this Blended 
Intensive Programme (BIP) combines virtual exchange with physical mobility. There 
were three institutions involved in this project: Letterkenny Institute of Technology, 
Ireland, Hochschule Bremerhaven, Germany, and Malaga Healthcare College, 
Spain. The virtual exchange ran for five weeks from November to December 2021 
and the physical mobility took place in Bremerhaven in the first week of April 2022. 
Twenty-seven participating students from diverse academic backgrounds engaged 
in shared tasks during the virtual exchange on a Blackboard platform, hosted by 
Letterkenny Institute of Technology and were awarded two ECTS credits, which was 
embedded into their local curricula. Content was delivered using both synchronous 
and asynchronous tools. The topic was global citizenship and students collaborated 
in multicultural teams to create presentations on their chosen topics. In addition to 
this team work, they also shared their individual insights into the course content by 
posting to a discussion forum in Weeks 1, 3, and 4. 


Keywords: blended mobility, virtual exchange, global citizenship. 
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mobility, before describing the specific details of this BIP. It will outline both 
virtual and physical elements of the programme, and describe student profiles, 
aims, and learning outcomes and show how the insights of prior studies in the area 
have informed the design and implementation of the current project. 


2. Blended mobility 


The concept of blended mobility, while in its infancy, has already generated interest 
among some scholars. Essentially, blended projects combine physical mobility 
with some element of virtual exchange; this blend of physical and virtual comes in 
various iterations. It is not surprising that the new Erasmus programme 2021-2027 
has prioritised blended mobility, given the manifold educational benefits it can 
provide (Helm & O’Dowd, 2020). While recognising these advantages, scholars 
are anxious to emphasise the challenges to be overcome when embarking on such 
projects. Purg, Sirok, and Brasil (2018) discuss the transformative effect of blended 
mobility in the context of the Master’s programme in Media Arts and Practices 
which ran from 2011 until 2014 in four EU countries; in particular, they highlight 
the intercultural value added to the learning experience (Purg et al., 2018). 
Welzer, Escudeiro, Druovec, and Holbl (2018) also acknowledge the positive 
impact of such programmes, as a way of internationalising the education system 
in the context of the AIM project which ran from 2016 until 2018 and involved 
ten EU countries. Nonetheless, they identify some of the intercultural challenges 
associated with blended mobility and propose a toolkit for dealing with them. 
Helm and O’ Dowd (2020) emphasise that learning pathways need to be carefully 
designed, so that students can get the maximum benefit from such programmes 
(Helm & O’ Dowd, 2020). These insights informed the development of the blended 
mobility programme described here. 


3. Virtual exchange and physical mobility 


While the preliminary project conducted in November 2020 was supported by 
Unicollaboration’s Moodle platform, the virtual element of this BIP was housed 
on Letterkenny Institute of Technology’s (now ATU Donegal) Blackboard 
website. The virtual-only exchange had created an opportunity for transnational 
collaboration during socially distanced Covid times (Carthy, 2021b); however 
the current programme adds another dimension to this with the physical mobility 
visit. The five-week virtual learning pathway extended from 8 November until 
12 December 2021. Malaga and Bremerhaven participants were given guest access 
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to this platform and completed a GDPR form, stating that they would abide by 
Letterkenny’s privacy policy regarding personal data. This was a vital preliminary 
step in setting up a learning environment with social presence for students to share 
and engage. Another important feature of the project, which had already been 
highlighted in the pilot project as a creator of social presence, was the assessment 
strategy (Carthy, 2021a). All set tasks involved some element of collaboration or 
sharing among students and were weighted as follows: 


e discussion forum: 40%; 
* team presentation: 40%; 
¢ synchronous sessions: 20%. 


In the discussion forum in Week 1, students had to introduce themselves to the 
community by posting a video/photo or text and respond to at least two of their 
fellow participants’ posts. In Weeks 3 and 4, students had to post a 50-word 
reflection on the synchronous session in English, Spanish, or German, and respond 
to at least one of the other posts. 


For the group presentation, tutors formed multicultural teams, ensuring a good mix 
of cultural background and gender. Once formed, groups had to select a leader, agree 
on a topic (from six possible options), and plan to deliver their presentation, with 
each member contributing two/three slides to the overall PowerPoint presentation. 


Finally, students were awarded 20% for their attendance at three synchronous 
sessions in Weeks 1, 3, and 4. These synchronous sessions were carefully managed, 
with a maximum of ten students per session and a good mix of nationality and 
gender. Week 1 concentrated solely on ice-breakers and games, to encourage 
maximum participation and engagement; whereas Weeks 3 and 4 focused on 
intercultural topics which would equip students with the skills necessary to carry 
out their teamwork. 


The physical element of the BIP, which took place in Bremerhaven, the designated 
host institution, was a week-long stay from April 4-10, some four months after the 
virtual element was completed. Prior to their departure, Malaga and Letterkenny 
students had to complete Learning Agreements and take out Covid insurance (in the 
event that they might contract the virus while in Bremerhaven). Students attended 
workshops in the mornings and went on sightseeing trips in the afternoons, with 
Thursday given over to a full day outing to Bremen. The workshops provided 
opportunities for participants to apply the intercultural concepts introduced during 
their synchronous sessions in Weeks 3 and 4 of the virtual exchange. On the 
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final day they shared their reflections on the workshop and did presentations in 
multicultural teams. 

4. Student profile and global citizenship 

In keeping with the priorities set out in the Erasmus programme guide, the student 
profile in this BIP was quite diverse, including ‘all types of students from all 


backgrounds, study fields and cycles”. 


Figure 1. Cultural and academic diversity 


Figure | (above) illustrates multicultural and academic diversity within BIP 
participants, with six nationalities represented in the Bremerhaven cohort, 
two nationalities in each of the Letterkenny and Malaga cohorts. However, the 
academic diversity is most noticeable in the Bremerhaven cohort, with a total of 
eight disciplines represented. Both Letterkenny and Malaga participants are from 
more homogenous backgrounds, by comparison. 


2. For more information see Erasmus programme guide: https://erasmus-plus.ec.europa.eu/programme-guide/part-b/key- 
action-1/mobility-project-for-higher-education-students-and-staff 
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This wide range of academic disciplines and multicultural diversity among 
participants provided a rich resource for the topic of global citizenship. The 
programme aimed to equip students with the following: 


* anunderstanding of collective identity, particularly intercultural sensitivity 
and respect for diversity; 


¢ an ability to think critically about global challenges and opportunities; 


¢ interpersonal and digital skills, with an aptitude for networking in 
multicultural settings; and 


¢ an ability to engage collaboratively to create a more sustainable world. 


5. Conclusion 


The pedagogical benefits of blended mobility programmes are quite evident in 
both the background literature and the BIP described above. Not only do these 
programmes have the potential to develop 21st century skills, but also intercultural 
and interpersonal communication skills so important in today’s multicultural 
workplace. However, as highlighted in prior studies, adequate design and 
management of both virtual and physical elements plays a pivotal role in their 
success. For virtual platforms, a sense of social presence on the learning pathway 
should ensure that students will engage and interact, thereby creating a dynamic 
learning community. This sense of community will be further enhanced when 
participants meet face-to-face for the physical element of the BIP. Further research 
into the learning experience of BIP participants will serve to inform future blended 
mobility programmes and ensure that more students from diverse academic 
backgrounds and nationalities have an opportunity to avail of them. 
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Evaluation of the DDL training workshop 
for in-service Chinese language teachers 


Li-ping Chang! and Chun-Ting Chou? 


Abstract. In this study, we held a 12-hour workshop focused on training the Data- 
Driven Learning (DDL) approach for in-service Chinese teachers aiming to implement 
this pedagogy in Mandarin Chinese classrooms in the future. We analyzed data from a 
postworkshop questionnaire to understand how the individual-level traits of Chinese 
teachers (such as their experience, ability, and motivations) relate to their integration of 
corpus-based methods. The questionnaire results indicate that teacher confidence in using 
the corpus is critical for the future application of DDL in teaching Chinese as a second 
language. Teachers who use corpus-based pedagogies are more positive at designing and 
guiding DDL activities and are more willing to practice DDL in language classes. 


Keywords: data-driven learning, Chinese language, teacher training, questionnaire, 
corpus. 


1. Introduction 


The DDL approach has been widely embraced in second language curricula in 
Wester countries for many years (Boulton & Tyne, 2013). However, corpus-based 
pedagogies, such as DDL, are recently gaining traction in Chinese as a Second 
Language (CSL) classrooms (Smith, 2018; Wang, Hsu, Long, & Liles, 2020). To 
facilitate the implementation of this method, studies have specified that language 
teachers should be acquainted with the basic concepts of corpora, skills of using 
corpora tools, knowing how to interpret corpora data, and knowing how to generate 
corpus output into their teaching specialty, etc. (Callies, 2019; Mukherjee, 2006). 
However, for CSL teaching, it is more challenging to familiarize teachers with 
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corpus consulting techniques, because the lack of clear boundaries between Chinese 
characters and words can make applying the DDL more difficult (Smith, 2018). 
Research has also indicated that the individual-level characteristics of teachers 
(for example, experience, motivation, attitudes toward corpus-based pedagogy) 
were related to their probabilities of using the corpus approach in the classroom 
(Chen, Flowerdew, & Anthony, 2019). To enhance future DDL teacher training 
outcomes, we will describe the design of our current training content and discuss 
our postworkshop questionnaire analysis. 


Zz: The DDL teacher training workshop 


The workshop, led by a university professor who specialized in corpus linguistics, 
was divided into four weeks with three hours of lectures per week using Google 
Meet. To increase the possibility and relevant skills of CSL teachers to adopt the 
DDL approach, the workshop mainly focused on teaching (1) basic concepts of 
Chinese corpora (e.g. word segmentation and part of speech); (2) skills in consulting 
Chinese corpora, particularly the largest Mandarin corpus in Taiwan, Chinese of 
Contemporary Taiwanese Mandarin; (3) the corpus query tools, Sketch Engine and 
AntConc; and (4) skills to analyze corpus data and generate pedagogical materials. 
Seventy-nine teachers (70% with a master’s degree) participated in the workshop, 
with an average teaching experience of 9.8 years. Fifty participants finished the 
training and completed the final assessment of the workshop, which required them 
to design teaching materials and plans for a 50-minute DDL activity. 


3. Gathering and analyzing data 
from postworkshop questionnaire 


After the last lecture, 50 of the 79 participants completed a postworkshop 
questionnaire. The questionnaire was inspired by Lenko-Szymanska (2015); it 
comprised four open-ended questions to investigate the views of the participants in 
the workshop and 39 Likert-scale questions, of which 36 questions covered seven 
variables that reflect participants’ traits, with three to 14 questions for each variable. 
These variables were self-evaluations of their (1) motivations for participating in the 
workshop, (2) prior knowledge of corpora, (3) skills toward using corpora after the 
workshop, (4) perceptions of adopting corpora in teaching, (5) skills to implement 
and design DDL activities, (6) willingness to adopt corpus-based methods in future 
teaching, and (7) evaluation of the workshop. These variables and the participants’ 
teaching experience and educational background form nine variables that reflect 
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the traits of the CSL teachers. To reveal the relationship between the personal 
characteristics of the CSL teachers and the possibility of adopting corpora in 
teaching to improve the DDL teachers’ future training, questionnaire responses 
were analyzed using a Spearman’s rank-order correlation test in RStudio software. 
Before the correlation test, the scores for the questions under each variable were 
averaged. The test results of the relevant variables are shown in Table |. The 
numbers in the center of the table are their p value (rho), and if there is any asterisk 
with them, the two variables have a correlation with significance. 


Table 1. Test results of relevance variables to CSL teachers’ use of corpus-based 


pedagogy’ 
1 2 3 4 5 6 7 8 9 
1 1 
24 40** 1 
2) 33* 21 1 
4 23 37** 31* 1 
5 -.12 22 12 A4OFEE | 
6 -.42* .08 -.13 29 eee | 7 
Wl .04 23 24 .28* J5QERE | SORE 1 
8 .00 -.15 06 .36* AS** 16 23 1 
9 14 15 A3** 22 .29* 24 A7*** | 16 1 
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i] 
a 
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a 


Variables that reflect the traits of CSL teachers are represented 
by Arabic numerals, which are as follows. 


* 1. Teaching experience (less than | year: 4%, 1-3 years: 14%, 3-5 
years: 16%, 5-10 years: 24%, more than 10 years: 42%). 


¢ 2. Educational background (bachelor’s degree: 24%. master’s degree: 70%. PhD: 6%). 
¢ 3. Motivation for participating in the workshop (mean score: 3.75/4). 
¢ 4. Prior knowledge of corpora (mean score: 3.03/4). 


¢ 5. Self-assessment of their skills toward using corpora af- 
ter the workshop (mean score: 3.09/4). 


* 6. Perceptions of adopting corpora in language teaching (mean score: 2.96/4). 


7. Self-assessment of the skills to implement and de- 
sign DDL activities (mean score: 3.14/4). 


* 8. Willingness to adopt corpus-based methods in future teaching (mean score: 2.75/4). 


¢ 9. Evaluation of the workshop (mean score: 3.36/4). 


* means p-value <.05; ** means p-value <.01; *** means p-value <.001. 


3. The number presented in the table is p value (rho), which represents the correlation between the two variables, ranging from 
1 to -1. Here, 1 represents the complete positive correlation between the two variables; -1 represents that the two variables are 
entirely negatively correlated; 0 means that the two variables have no correlation. 
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The results indicated that the CSL teachers were more confident in their corpus 
skills after training, more motivated to participate in the workshop, more willing to 
implement the DDL approach in future classes, and less resistant to using corpora. 
Furthermore, CSL teachers with prior knowledge of corpus-based pedagogy were 
more motivated to participate in training, had more confidence in their skills to 
operate corpora and implement DDL activities after training, were more willing 
to apply corpus-based pedagogy in the future, were less resistant to adopting 
corpora, and had a higher training evaluation score. The teachers’ evaluation of the 
workshop and responses to the open-ended questions were positive. Many teachers 
responded that what the workshop had taught was of practical use to their teaching. 
This is compatible with research demonstrating that teacher anxiety and confidence 
in using corpus are major factors inhibiting the application of DDL approach (Lam, 
2000; Rémer, 2009). 


4. Conclusions 


Considering the positive feedback from CSL teachers, we believe that training 
courses benefit in-service teachers, reducing their anxiety about technology, and 
enhancing their confidence in integrating technological methods into teaching. 
By teaching corpus knowledge and guiding teachers to use corpora, our study 
trained in-service CSL teachers in DDL teaching, which is rarely practiced in 
CSL. Our analysis of the post-questionnaire results indicates that teachers with 
greater prior knowledge of corpus and more confidence in their abilities are more 
likely to apply the DDL approach in future classrooms. These findings indicate 
the utility of training workshops. However, it is too early to assess the actual 
benefits of DDL for CSL learners because it is not yet used in classrooms, and 
more research is required. Our study is but a starting point for research on the 
DDL approach in CSL. 
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A study of language, culture, and identity in 
technology-assisted video presentations of international 
students learning Chinese as a second language 


Li Cheng! 


Abstract. This paper reports a study in a technology-assisted Chinese teaching 
context where one major teaching objective is to encourage international students 
to participate in speech contests with the theme of ‘Tell China, Tell Your Stories’. 
Results showed that through four weeks’ preparation, all the students had a more 
robust method for presentation delivery and video making than before the project. 
Moreover, meeting conventional presentation standards was not only an issue of 
‘language learning’, but also an issue of identity construction and negotiation. 
Finally, all participants had constructed positive ecocultural identities which 
encouraged respect for individuals, collaborative working, and a sustainable society. 


Keywords: language and culture, identity, presentations, teaching Chinese as a 
second language. 


‘A: Introduction 


In the field of language learning, researchers of language, culture, and identity 
maintain that students develop ‘a sense of ownership’ of the target language that 
will ‘enable them to better negotiate their current circumstances and articulate a 
vision for the future’ (Norton & Gao, 2008; Norton & Toohey, 2011). Due to the 
impact of COVID-19, many educational exchanges in China have changed from 
face-to-face interactions to a fully online format, e.g. virtual conferences, online 
education and training platforms, online discussion communities, etc. 
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The purpose of this study is to explore the construction of identity from the 
sociocultural linguistic perspective. The research is guided by the theories of 
language socialization (Schieffelin & Ochs, 1986) and identity negotiation theory 
(Ting-Toomey, 2015). According to language socialization, learners are socialized 
in a community of practice. In this study, identity is defined as “an individual’s 
multifaceted identities of cultural, ethnic, religious, social class, gender, sexual 
orientation, professional, family/relational role, and personal image(s) based on 
self-reflection and other categorization social construction processes” (Ting- 
Toomey, 2015, p. 418). This sense of identification can be divided into four levels: 
individual, social, national, and ecocultural. 


Apart from identity, two concepts of culture and story are also defined. Culture is 
the shared values, beliefs, and practices that can be interpreted in the meanings 
constructed by the participants in the local, national, and global contexts 
(Croucher, Sommier, & Rahmani, 2015). In this sense, the analysis of culture and 
communication from the ecological perspective can be viewed as “an engaged 
form of critical inquiry” featuring “discourse analysis of a wide range of texts” 
since “[l]Janguage, culture, human cognition, stories and texts play a role in 
human ecology to the extent that they influence human behaviour, and hence the 
ways that humans interact with each other and the larger natural world” (Stibbe, 
2015, p. 8). According to Stibbe (2015), discourses are the “stories in the minds 
of multiple individuals across a culture” (p. 6) and involve an element of cultural 
theme. 


The two research questions are as follows: (1) What are the characteristics of the 
presentations? (2) How do the international students construct their identities in 
the presentations? 


2. Method 


This study (June-November of 2020) took place in the International Chinese 
Training Center at a university in Beijing. ‘Language, Culture and Communication: 
Tell China, Tell Your Stories’ is a pedagogical project aiming at helping 
international students improve their Chinese, share and appreciate cultural 
similarities and differences, and increase cultural awareness through participating 
in Chinese speech contests at three levels, including a university-wide event (i.e. 
‘International Forum on Intercultural Communication’), a national event (i.e. 
‘Chinese Classics Recitation and Presentation Contest’), and an international event 
(i.e. ‘ZUI Confucius Institute, Chinese Culture’, Figure 1). 
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Figure 1. ‘Language, culture and communication: tell China, tell your stories’ 
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International Forum Chinese Classics 
on Intercultural Recitation and 
Communication Presentation Contest 


A 


university event national event 


The participants were 15 international students from ten countries along the Belt 
and Road. They had passed the Chinese proficiency test HSK-5. They spent four 
weeks getting prepared for the speech events under the guidance of a team of 
Chinese instructors. 


ZUI Confucius 
Institute - Chinese - 
Culture 


international 
event 


Data include 26 video presentations submitted to three online events mentioned 
above, a questionnaire survey, ten semi-structured interviews, a pre-test and a post- 
test of the participants’ technology use based on the research by Kim, Kim, Khera, 
and Getman (2014), and reflective teaching notes. All the presentations were 
transcribed (26,311 Chinese characters) and then analyzed. 


The questionnaire was adapted from those of Brown (2009) and Littlewood 
(2001) and evolved after being piloted with a different student group. It consisted 
of 30 items in three sections: views of authority (Cronbach’s a=0.727), students’ 
identities (Cronbach’s a=0.713), and intercultural competence (Cronbach’s 
a=0.732). 


The questionnaire was distributed to the participants in mid-December of 2020 
and 15 (100%) replies were collected. All the interviews were recorded and then 
transcribed. 
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3. Results and discussion 


Discussion of the results in this section attempts to answer the two research 
questions listed above. 


3.1. Characteristics of the presentations 


Analysis of the presentations shows two characteristics: technological support and 
beneficial discourses. First, results of the pre-test and the post-test show that all 
participants had a more robust method for presentation delivery and video making 
than before the project. The reflective teaching notes gave supporting evidence that 
the participants had made great progress in such presentation skills as ‘Chinese 
language communication’, ‘Non-verbal communication’, and ‘Visual aids’. When 
it comes to ‘Visual aids’, the students used a variety of technological tools to help 
them present and make presentation videos including virtual reality classrooms and 
video editing software. 


The second feature of the presentations is beneficial discourses identified based on 
discourse analysis. Discourses can be divided into three categories: destructive, 
ambivalent, and beneficial. Despite their different cultural backgrounds, all 
participants shared their stories and their understanding of Chinese culture 
(Table 1). As can be seen from Table |, the top three ecological viewpoints are 
“Cultural Differences and Shared Prosperity’, ‘Human and Nature’, and ‘Chinese 
History and Classics’. 


Table 1. Viewpoints in the presentations 


Number of Viewpoints Percentage 
presentations 

10 Cultural Differences and Shared Prosperity 38.5% 

4 Chinese History and Classics 15.4% 

2 Digital Humanities 7.7% 

6 Human and Nature 23.1% 

4 Sustainability and Intangible Culture 15.4% 


3.2. Students’ identities 


Presented below (Table 2) is the descriptive analysis of the questionnaire data 
related to authority, identity, and intercultural competence. 
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Table 2. Descriptive analysis of the questionnaire data 


Mean Standard Deviation 
Views of authority 4.36 0.93 
Students’ identities 4.02 0.82 
Intercultural competence 3.98 0.78 


Results indicate that the students’ identities were closely related to their views of 
authority. This was supported by the interview data that the participants approached 
collaborative learning and seeking guidance from the teachers with positive 
attitudes. Moreover, the participants regarded the teacher’s role as the authority 
of knowledge transmission. During the four week’s training, although they used 
the virtual reality classrooms and video editing software quite often, most of them 
relied heavily on the teachers’ help and focused on language learning rather than 
creativity and critical thinking. Furthermore, 13 students mentioned that they had 
improved their intercultural competence. As Ruisheng said, 


“participating in the speech contests has helped me appreciate the beauty 
of the Chinese poems and make connections with the current situation. 
‘No clouds will block the sun and no way can the winter slow down the 
pace of the spring’. Salute to all those who have joined the fight against 
the pandemic! ((smile and confident))” (Interview, 2020-12-21). 


From an ecological learning standpoint, Ruisheng positioned herself both in the 
local context of the speech event and in the global context of fighting against 
COVID-19. The analysis of the students’ identities shows the important role of 
identity in language learning contexts (Klimanova & Dembovskaya, 2013). 


4. Conclusions 


In this study, the researcher explored the dynamics of identity in a Chinese teaching 
context. Results show that the participants have not only improved their skills in 
technology and presentations but also enhanced their cultural sensitivity. More 
importantly, they had constructed a variety of identities in the local and global 
contexts, which was reflected in the students’ learning activities, including 
presentation training activities and the speech contests. 


The findings provide supporting evidence for the complicated relations between 
language learners and their learning communities. As learners of Chinese take 
greater ownership of the language and redefine the learning communities, online 
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and offline, researchers and educators should reconsider the role of economic, 
sociocultural, and political factors in learners’ identity construction and negotiation 
in natural settings. 
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Abstract. Modern language learning applications have become ‘smarter’ and 
‘intelligent’ by including Artificial Intelligence (AI) and Machine Learning (ML) 
technologies to collect different kinds of data. This data can be used for analysis on 
a microscopic and/or macroscopic level to provide granulation of knowledge. We 
analyzed 1,213 French language learner data over a 30-day period, publicly available 
from Duolingo, to compare the progression of individual learners (microscopic 
granularity) and large groups of learners (macroscopic granularity). Using network 
modeling, we compared patterns of individual learners against one another and that 
of a learning community and determined what groups of learners typically practice 
across communities. Preliminary results suggest how applications for L2 learning 


can be designed to create an optimal path for learning. 


Keywords: artificial intelligence, granularity, language learning journey, machine 


learning. 


1. Introduction 


In the past ten years, an increasing number of language learning applications have 
become ‘smarter’ and ‘intelligent’ by including features from both AI and ML. Both 
technologies enable collecting different data types for analysis on a microscopic 
and/or macroscopic level. Such analysis is called granularity, with microscopic 
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granularity focusing on individual learners while macroscopic granularity compares 
large learner groups. This article explores both granularities and provides insights 
into a language learning journey by analyzing a sample of French language learner 
data. We focus on identifying and contrasting common patterns of learning behavior 
for insight into learners experiencing either exceptionally good or poor progress. 


Many learners engage and possibly re-engage with a language learning program 
over time, in the pursuit of learning a target language (L2). In the majority of 
‘smarter’ or ‘intelligent’ language learning applications, AI/ML technologies learn 
about, e.g. a learner’s errors, engagement, time and program usage, mistakes in 
specific exercises, and so forth. Programs with AI/ML technologies collect data on 
individual learners performing similar actions over time yet resulting in varying 
levels of language competency. Insights are achieved through learning analytics, 
or microscopic granulation, to influence a participant’s motivation and improve 
learner behavior to keep the learner actively engaged (Greller, Ebner, & Schon, 
2014; Hai-Jew, 2014). Results from macroscopic granulation can utilize all 
sources of learners’ actions to create an optimal path for learning (Tang, Peterson, 
& Pardos, 2016). Such granularity helps to design lessons in learning systems 
to become more digestible (Jasnani, 2013), i.e. less challenging in terms of the 
required competences, e.g. time management, learner’s self-regulated learning, 
and self-organization (Lackner, Ebner, & Khalil, 2015). 


2. Method 


We used data generated by 1,213 French language learners over a 30-day period, 
publicly available from Duolingo (Settles, 2018), to demonstrate micro- and 
macroscopic granulation; chosen solely for the reason that this data is freely 
available. Each learner had a unique, anonymized ID with their session data 
including country of access, days in the program, time taken to complete a task, 
Part-Of-Speech (POS) tags, and whether the learner produced a correct answer. 
Three session types were recorded: /esson, practice, and test with three different 
activities, listen, reverse tap, and reverse translate. Reverse translate gives a 
learner a question in L1 to translate into L2 whereas reverse tap requires learners 
to construct an answer to a question in L1 using a small set of words (including 
distractors) in L2 (Settles, 2018). Most learners attempted all activities, 
however typically used two of three session types. Reverse translate was used 
by all learners while individuals selecting two activities predominantly also 
selected listen. Nearly all learners used /esson with several using test. A popular 
combination was /esson and practice. 
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We created networks of node-edge pairs from the data. For microscopic granularity 
we investigated co-occurrence networks of an individual learner’s frequency of 
exposure to POS component combinations (NOUN, VERB, DET, etc.) and the 
vocabulary breadth encountered. For macroscopic granulation we grouped 
by country and created signed networks capturing correct and incorrect POS 
combinations, to provide insight into key lexical combinations learners found 
challenging or easier to grasp. The Jaccard index (Agresti, 2002) for network edge 
set similarity identified common elements between networks. 


A mixed methodology compared individual performance against a community. The 
benefit of this approach is the formation of networks creating a multidimensional 
learning structure, even if learners do not engage directly with each other. These 
insights can benefit learners by gauging individual proficiency against common 
learning patterns of other individuals with a similar profile. 


3. Results and discussion 


Microscopic granulation (Figure |) compares the typical time taken to complete 
learning activities of two learners, where learner (a) interacted with all activities 
over 25 days in the program, completing 752 tasks, whereas learner (b) used 
reverse translate only over 27 days and completed 131 tasks. 


The POS graphs in Figure 2 indicate regular exposure to similar combinations 
(NOUN, VERB, DET, PRON) although learner (b) received greater exposure to 
ADJ, ADV, and INTJ components despite engaging less frequently. Vocabulary 
varied by learner. Figure 3 shows learner (a) predominantly practiced determinants 
le and un while learner (b) also practiced a third determinant, /es. 


Figure 1. Time taken to complete the learning activities for two learners 


° | -_ 


——_ 0 


listen reverse_tap reverse_translate 


Time Taken (s) 
Time Taken (s) 


reverse_translate 


(a) (b) 


60 


Using learner data from Duolingo to detect micro- and macroscopic granularity... 


Figure 2. Exposure to POS by learner. Heavier lines indicate more frequent 


exposure 


PUNCT 


PUNCT 


CONJ 


CONJ ADJ 
ADV INTJ 
PROPN ae PROPN 
(a) (b) 
Figure 3. Vocabulary exposure by learner. Heavier lines indicate more frequent 
exposure 
son singe 
merci vétements 
livre fe oy ad pouret merci Min 
chat — vig we as feames homme or chapeau 
hic 
a S chapeau animal sae oa = soupe 
ma es —_ ss 
= S 
chiens enfants 7 livre pomme Pewee 
r chien chip femmes 
lapin gargon huile 
fille serpent urie femme 
(a) b) 


Figure 4. Signed networks indicating correct answers in blue and incorrect 


answers in yellow 


(a) 


(b) 
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In Figure 4, learners (a) and (b) are compared using signed networks to capture 
POS combinations. The learners answered correctly on average (blue) and 
incorrectly on average (yellow). Although learners were exposed to different 
vocabulary (Figure 3) and training activities, the data indicated similar proficiency. 
It could be postulated that learner (a) had less prior language exposure, while learner 
(b) engaged with the learning program to refresh their knowledge. The Jaccard 
index reported 71% commonality between the learners, supporting similarities in 
learning outcomes. 


Results from macroscopic granulation can potentially be implemented as part of 
a language learning platform to provide a sophisticated learning mechanism for 
learner agency, to enable a language learner to compare their progress with that 
of their peers. For example, macroscopic granularity may alert a learner to limited 
variability in exposure to key language components when compared to their peers, 
which would encourage the learner to request more advanced learning modules. A 
mixed methods approach (Figure 5) compares learner (a) with all learners from the 
same country of access (b), to inform expected proficiency of an individual, based 
on country of access. Learner (a) is typically faster with reverse tap than other 
learners from the same country, however, takes typically longer with activities 
such as reverse translate, with extreme values comparable to those of the learner 
community. 


Figure 5. Time taken to complete the learning activities from an individual 
learner (a) and all learners from the same country of access (b) 
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Time Taken (s) 
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In Figure 6, learners in countries (a) and (b) are compared. Country (a) learners 
are typically quicker with reverse tap however typically slower otherwise. Such 
insights could inform the language learning platform how to adjust content 
and challenges such that learners from country (a) could be encouraged to 
practice listen and reverse translate, whereas learners from country (b) could 
be encouraged to engage with reverse tap tasks. These recommendations could 


62 


Using learner data from Duolingo to detect micro- and macroscopic granularity... 


be implemented within the language learning tool, thereby allowing learners 
the opportunity to take responsibility along their language learning journey by 
evaluating their own progression against a community of peers of interest. 


Figure 6. Time taken to complete the learning activities between learners 
accessing the program from countries (a) and (b) 
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4. Conclusions 


This article explored microscopic and macroscopic granularity to demonstrate 
that learner data accessed at individual and group levels can provide insights into 
assessing an individual’s learning journey, to allow supportive learning advice that 
can be auto generated when combined with more sophisticated iterations of this 
analysis. Having insight at a micro-granular level allowed a comparison of learners 
to uncover individual progression relative to engagement with the language learning 
platform. Macro-granular level analysis allowed for a complementary comparison 
at a community level, to allow assessment of the individual’s performance against 
a community of peers of interest, as well as the ability to compare communities 
with one another. 


Limitations of the current study include the investigation of a single data set 
comparing English and French users and the inability to characterize learners 
beyond their country of access, which limits the potential to compare community 
structures. Future research already underway considers the exploration of more 
than one data set to provide granularity across multiple languages of interest. To 
support learner characterization, open-source applications and those supported 
by crowdsourcing are encouraged to make richer datasets publicly available 
to researchers who are interested in designing new, or improving existing, 
applications. 
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the evolution of research strands 
in influential CALL papers 
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Abstract. To trace the evolution of research strands in published Computer Assisted 
Language Learning (CALL) papers over time, a corpus of 426 highly-cited papers 
in four major CALL journals (ReCALL, CALL, LL&T, and CALICO Journal) was 
compiled and coded using NVivo 12. All identified aspects of technology-mediated 
language learning and teaching were collected to produce a comprehensive list of 
690 recurring research strands and then, by adopting a constant comparison method, 
were merged to form 119 unique research strands. The top 10 alone represent almost 
half of all research strands: Computer-Mediated Communication (CMC), writing, 
vocabulary, feedback, evaluation, learning environment, telecollaboration, design, 
speaking, and grammar. This paper provides the rationale for the study along with 
the methodology for data collection and analysis, with a particular focus on the 
dominant and intermediary strands to inform future CALL publications. 


Keywords: CALL research, CALL evolution, research strands, corpus analysis. 


1. Introduction 


Since its inception, CALL has been widely adopted to support learning, assess 
learners’ knowledge of language skills, and assist with data collection/analysis for 
both classroom-related and beyond-the-class inquiry. With the rapid development 
of CALL over the years, CALL-oriented journals have emerged and expanded to 
invite their readers to keep up with the pace of technological advances through 
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academic Research Articles (RAs), attesting to the influence of CALL research 
(Gillespie, 2020). 


These CALL papers, combined with other academic genres such as books and 
book chapters, master’s theses, and doctoral dissertations, as well as conference 
proceedings, have had a substantial role in shaping the history of CALL research. 
To trace part of that history, this narrative review charts the evolution of research 
strands in highly-cited CALL RAs. The main objective of the study is to 
concentrate on the past and present of CALL research strands, build on their 
diversity, and accordingly propose avenues for the future of researching CALL. 


2. Method 


The pool of data consists of all 2,397 RAs published in English in four major 
CALL journals: CALL, ReCALL, LL&T, and CALICO Journal, from their first 
appearance up to and including 2019. A novel approach pinpointed the top 15% 
of highly-cited RAs based on Google Scholar (for more details see Choubsaz, 
Boulton, & Jalilifar, 2020; Choubsaz, Jalilifar, & Boulton, 2021), providing 
greater focus in the final corpus of representative high-impact CALL papers. 
A coding scheme was then developed for recording the existing research 
strands. As extracting, coding, and merging research strands involves a degree 
of interpretation, we followed Riazi, Shi, and Haggerty (2018) in opting for a 
data-driven thematic approach, coding the data by drawing on the actual words 
of the authors as much as possible. To add objectivity and consistency, 10% of 
the RAs were coded in two separate rounds by two independent coders for an 
inter-rater r of 89.8. The analysis was also complemented by substantial use of 
computer-assisted corpus analysis tools (i.e. NVivo 12 and AntConc) to facilitate 
the coding of the research strands. 


3. Results and discussion 


The diverse aspects of CALL and the multiple research strands in each study 
posed major challenges. Therefore, to attain a high level of granularity, an 
exhaustive list of research strands was compiled, including all identified aspects 
of language teaching and learning (initial coding); 690 recurrent research strands 
were collected and fed into NVivo 12 and AntConc for a detailed analysis. The 
strands were later manually merged by adopting a constant comparison method 
(focused coding). By taking account of the technologies adopted as a separate 
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context-related sub-theme, the strands identified in our study truly reflect the 
primary research areas in high-impact CALL RAs and are not overrepresented 
by technology-related themes; 119 unique research strands were thus identified. 
Due to space constraints, only the top ten and the 27 intermediary strands are 
discussed here as they have appeared in at least seven studies over almost four 
decades of researching CALL. 


Dominant research strands. As illustrated in Table |, the top ten characterize nearly 
half of all the research strands over the four time periods under investigation. 


Table 1. Top 10 CALL research strands across four time periods 

3%) 1983-1989 | 1990-1999 | 2000-2009 | 2010-2019 | Total 
CMC - 10 34 24 68 (9.8%) 
Writing 2 6 21 16 45 (6.5%) 
Vocabulary 1 8 14 10 33 (4.7%) 
Feedback 1 7 7 ial 26 (3.7%) 
Evaluation 5 10 4 6 25 (3.6%) 
Learning environment | 1 8 6 10 25 (3.6%) 
Telecollaboration - - 11 13 24 (3.4%) 
Design 5 6 8 4 23 (3.3%) 
Speaking 1 4 6 12 23 (3.3%) 
Grammar 3 13 5 2 23 (3.3%) 
Total 19 72 116 108 315 (45.6%) 
Figure 1. The evolution of top research strands across four time periods 
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The most dominant research strands are CMC (9.8%), writing (6.5%), and 
vocabulary (4.7%). One of the main findings is that although the top ten strands 
have been the focus of CALL research over time, some of them seem to have lost 
popularity in the last decade, a fact that is noticeable with the decline in entries 
from 116 to 108 RAs during 2010-2019 (see Table | and Figure | above). The other 
findings can be the arrival and increased popularity of telecollaboration in the last 
two decades and the constant rise of speaking. 


Intermediary research strands. The 17 mid-frequency strands (Table 2) constitute 
almost one third of the total (213, 30.8%); together with the top ten, they 
represent 76.4% of the overall data set. However, there is a major difference: 
with the exception of reading, teacher education, and studies focused on error 
analysis/correction/diagnosis, which see an increase from the second to third time 
period and then a decline in the fourth, most of the research strands in this category 
have remained popular or increased over time. 


Table 2. Intermediary CALL research strands across four time periods 


1% — 2.9%) 1983-1989 | 1990-1999 | 2000-2009 | 2010-2019 | Total 
Task 1 1 10 9 21 (3%) 
Technology 2 5 2 9 18 (2.6%) 
effectiveness 
Listening 2 3 6 15 (2.1%) 
Proficiency 1 1 5 8 15 (2.1%) 
Learner variables1 | - 1 4 10 15 (2.1%) 
Teacher education | - 1 8 5 14 (2%) 
Reading 2 3 7 14 (2%) 
Perception - 1 4 14 (2%) 
Learner autonomy | - 1 1 11 13 (1.8%) 
Error2 1 8 2 2 13 (1.8%) 
Technology - - 3 8 11 (1.5%) 
affordances 
Culture - 3 1 5 9 (1.3%) 
Pronunciation - 4 2 3 9 (1.3%) 
CALL program 4 4 - 1 9 (1.3%) 
Glosses - 1 5 2 8 (1.1%) 
Learning 1 2 2 3 8 (1.1%) 
strategies 
CALL research - 2 3 2 71%) 
Total 14 41 65 93 213 (30.8%) 
Note. 1. Behavior, attitudes, engagement, self-efficacy, performance, socialization. 


Note. 2. Analysis, correction, diagnosis. 
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Identifying 119 research strands (out of 690 recurrent strands in our collection) 
complements similar syntheses like that of Lim and Aryadoust (2021) who 
established seven major research clusters in 11 CALL journals. However, our 
synthesis is distinctive in offering a bird’s-eye view of the research strands 
reflecting all identified aspects of language teaching/learning. 


4. Conclusions 


To conclude, this study charts the evolution of research strands in highly-cited 
CALL papers over time. Similar to the findings of Gillespie (2020), the most 
central strands over four decades of researching CALL have been CMC (9.8%), 
the four language skills, along with the essentials of first/second language learning 
such as vocabulary (4.7%), feedback (3.7%), evaluation (3.6%), telecollaboration 
(3.4%), design (3.3%), and grammar (3.3%); in other words, classroom-related 
research areas plus physical and virtual learning environments still inform the 
research strands after four decades. 


Additionally, the increasing breadth of research strands informing the field can be 
observed in the most recent time periods, when new strands such as collocations, 
learning outcomes, and identity practices have emerged and been addressed in 
special issues of CALL journals in the form of position and review papers. In 
the light of narrative review, we found that under-researched strategic research 
strands such as CALL and ethics and cultural CALL (Gillespie, 2020) need 
further investigation. 


Moreover, CALL researchers have recently tended to move toward sophisticated 
strands and interdisciplinary projects in the field of computer sciences, including 
eye-tracking technology and robot-assisted language learning. The shift toward 
such sophisticated strands necessitates the adoption of integrated methodologies, 
complex designs, and well-established theories so that CALL research can reflect 
its current interdisciplinary state (Hubbard & Colpaert, 2019). 
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Abstract. This study reports on the development of Parlure Games and explores its 
pedagogical affordances. Parlure Games is a multi-dialectal listening and decolonial 
conversation tool created to address the absence of variable speech (including speech 
markers associated with native speaker status, regional dialect, age, and race) found 
in the audiovisual material of adult French learners in Montréal, Canada. Parlure 
Games enables instructors to curate audiovisual content inclusive of different social 
and regional dialects, and supports learners in understanding variable speech while 
self-locating themselves in the process of learning a colonial language. 


Keywords: Parlure Games, ecological CALL, variability in L2 input, French L2 
learning. 


1. Introduction 


Nested within layers of macro-sized (national cultures, institutions) sociocultural 
ecosystems, the classroom is a microsystem where technology can transport 
learners from local to international contexts via Computer-Assisted Language 
Learning (CALL; Ramirez, Lafford, & Wermers, 2021). For language learners 
in urban settings, the ability to attune to local and international speakers is an 
important communicative skillset to develop for their social inclusion (Otsuji & 
Pennycook, 2011). However, the well-documented lack of social and regional 
variation found in language classrooms reflects an assimilative and colonial 
mindset toward learning (Macedo, 2019). Language learning engages multiple 
modalities (visual, aural, and emotional; Hardison & Pennington, 2021), and is 
enhanced when multiple speakers and varieties are encountered (Pisoni, Lively, 
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& Logan, 1994), meaning that classrooms should strategically curate a curriculum 
inclusive of local and international varieties to encourage multimodal interaction 
in the Target Language (TL). 


Parlure Games was created to address these issues, particularly the ‘invariable’ 
social speech markers (related to age, race, native speaker status, and regional 
dialect) found in the audiovisual material (TV shows, movies) that characterizes 
adult French Second Language (FSL) classrooms in Montréal, Canada (Chung & 
Cardoso, 2022). Parlure Games 1s a multi-dialectal listening and conversation tool 
designed to expose learners to variable TL speech and prompt critical conversations 
about learning a colonial language. Situated within stages one and two of Cardoso’s 
(2022) chronological framework for conducting interactive CALL research, this 
paper (1) describes the development of Parlure Games, and (2) explores the tool’s 
pedagogical affordances. 


2. Stage 1. Parlure Games: development 
of an ecological, variable CALL tool 


Ecological perspectives do not view language as input or output, rather language 
provides opportunities for learners to attune to their sociolinguistic environment 
through interaction with human and non-human agents (i.e. technology, signs), 
either enabling or weakening linguistic relationships (Lafford, 2009). These 
multimodal interactions enhance the “chorus of voices” that constitute learners’ 
communicative repertoires (Tarone, 2007, p. 842), and proficiency is measured 
by performing increasingly complex interactive tasks (Lafford, 2009). Ecological 
learning, therefore, focuses on how language is used in the nested ecosystems, 
which means thinking both locally and globally about language teaching and 
learning. Drawing on ecological systems theory, van Lier (2011) outlined ten 
characteristics of an ecological language learning classroom (Table 1) which is 
principled on supporting learners’ agency development as they self-locate their 
identities in the nested ecosystems. 


To explore the interconnected macro and microsystems of the linguistic landscape, 
learners require robust perceptual systems. Empirical evidence from laboratory- 
based High Variability Phonetic Training (HVPT) suggests that second language 
learning is enhanced when training includes: (1) multiple human voices; (2) 
natural speech; (3) at least five distinct talkers (e.g. each representing a unique 
dialect); (4) multiple exposures to the stimuli; and (5) immediate feedback using 
forced-choice identification tasks (e.g. choose answer A or B; Pisoni et al., 1994). 
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HVPT is consistently linked with greater perceptual acuity, more target-like TL 
pronunciation, and learning that generalizes to new speakers and novel linguistic 
features (Thomson, 2018). These five characteristics form the theoretical pillar for 
curating highly variable audiovisual content for the language classroom, and are 
operationalized in Parlure Games. 


3. Stage 2. Parlure Games: pedagogical affordances 


Variation is essential for robust language learning, and FSL classrooms are 
consistently criticized for the lack of regional and social variation in their curriculum, 
often favoring standardized varieties reflective of middle-class white speakers 
(Macedo, 2019). The use of mass media (e.g. TV shows, films) in the classroom 
contributes to monolingualism, as it often features invariable standardized speech 
from supposedly ‘unaccented’ speakers (Lippi-Green, 2012). Because invariability 
inhibits robust language learning, Parlure Games was designed to disrupt these 
monolingual orientations by sourcing its audiovisual material from non-mass media 
platforms (e.g. YouTube, TikTok). These online videos often offer captioning, 
which supports bimodal learning (i.e. simultaneously reading while listening). 


Figure 1. Interface of Parlure Games 


PARLURE GAMES 


Listening sessions 


9 Blue pins sociolects (city/province) 
9 Red pins regional dialects (country) 
9 Green pins international dialects 


A Parlure Games expedition includes local ‘road trips’ within a region (e.g. 
province, city), or international ‘transatlantic voyages’ to previously or 
currently colonized territories within the Francophonie (see Figure | above). 
One expedition is comprised of five destination points pinned onto a Google 
map, each containing a hyperlink to a publicly available video. Following the 
characteristics of HVPT training, each video represents the natural speech of at 
least five distinct talkers, and the playback setting allows for multiple exposures 
to the stimuli. After listening to each video, learner groups collectively answer a 
hyperlinked quiz (with automated feedback) related to content and social speech 
markers. Finally, group discussions are posed, prompting critical conversations 
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about learning a colonial language, facilitating learner engagement with the 
macro and micro realities of learning French, thus supporting an ecological CALL 
pedagogy. Table | illustrates how Parlure Games can promote an ecological 
language learning classroom based on van Lier’s (2011) framework. 


Table 1. 


Ecological 
pedagogy 


Definition 


Promoting an ecological language classroom with Parlure Games 


Parlure Games 


1. Relationality 


Examines the interconnectedness 
of linguistic systems from 
a micro/macro level 


Online map features 
audiovisual material of local 
and international varieties of 
Francophonie communities 


2. Context 


Language use is specific to 
sociocultural contexts involving 
in/animate interlocutors 


Geographical and sociocultural 
contexts are explored 

through online videos and 
discussed in groups 


3. Patterns/Systems 


Emerge from language use 
not prescriptive rules 


Exposure to authentic (i.e. 
non-mass media) speech 


4. Emergence 


Multimodal communicative 
repertoire is constantly 
re-organizing itself 


Audiovisual material 

and group discussions 
engage the multimodal 
communicative repertoire 


transformative education 


5. Quality Relevant to learners with real- | Content and discussion 
world implications that engage _| questions enables learners to 
both the intellect and emotions __| check their comprehension 

and express their opinions 

6. Value Cultivate a moral stance Discussing French’s global 
toward language learning reach raises ethical questions 

of learning a colonial language, 
cultivating a moral stance 
on users’ FSL learning 
7. Critical Examining institutional Supports learning how to 
Perspective structures and supporting perceive the language and 


critically discuss these 
perceptions with classmates 


8. Variation 


Encourages embodying a 
linguistic identity among the 
many varieties available 


Spotlights non-standardized 
varieties enabling learners to 
choose which linguistic identity 
they wish to embody/reject 


9. Diversity 


Promotes multilingual learning 
over monolingual approaches 


Reflects a multi-dialectal 
approach to learning over 
a standardized one 


10. Agency 


Inhibits or enables 
sociolinguistic movement 
through an ethical education 


Learmers traverse the globe 
listening to different speakers 
and discussing content, 
supporting perceptual learning 
and critical reflection 
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4. Concluding remarks 


The goal of an ecological CALL curriculum is activity-based learning around, 
with, and through the use of technology (Cardoso, 2022), providing learners 
with opportunities to hone their attunement skills, cultivate learner agency, and 
socially engage with the TL (van Lier, 2011). Parlure Games enables language 
instructors to curate an audiovisual curriculum inclusive of speakers from 
diverse social and regional backgrounds, and supports learners in navigating 
their multilingual environment while connecting them to ecosystems greater 
than themselves. 
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Pre-service teacher education and the integration 
of mediation, technology, and plurilingualism 


Giulia Ciaramita! 


Abstract. Although some research has been conducted on the importance of 
mediation in language learning and teaching (Dendrinos, 2006; Gonzalez-Davies, 
2020; Piccardo, 2012, 2020; Scarino, 2016), there is still scarce research on the 
integration of plurilingualism, mediation, and technology. Through qualitative and 
quantitative methodology, this paper investigates teachers’ abilities in Italy and 
Spain to integrate plurilingualism, mediation, and technology. A survey has been 
distributed in order to explore teachers’ attitudes towards the use of Information 
and Communication Technologies (ICTs) and their awareness of the definition and 
importance of mediation. Furthermore, some mediation tasks performed by teachers 
in which they had to integrate mediation, plurilingualism, and technology were 
analysed. 


Keywords: teacher education, mediation, ICTs, language learning, TEMPLATE 
project. 


ale Introduction 


Mediation can be considered as the act of language that makes “communication 
possible between persons who are unable [...] to communicate with each other 
directly” (Council of Europe, 2020, p. 34). This definition is provided by the 
CV, which also underlines the relationship between mediation and plurilingual 
and pluricultural competences. Mediation was already present in the Common 
European Framework of Reference for languages (CEFR), published in 2001 
(Council of Europe, 2001), but it was not sufficiently developed: the only two 
mediation strategies were translation and interpretation (Piccardo, 2020). The 
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Council of Europe (2015) decided to further develop the topic in the CV since 
nowadays societies are characterised by cultural and linguistic diversity and, 
therefore, the citizens need to be able to mediate. 


After the publication of the CV, some research has been conducted on the 
importance of mediation in language learning and teaching. Some academic 
research focuses on the presence and role of mediation in the CEFR (Piccardo, 
2012, 2020), while other studies have been conducted on translation as a mediation 
strategy (Gonzalez-Davies, 2020; Scarino, 2016). Other research focuses on the 
role of mediation in language certifications (Dendrinos, 2006) and some ICT tools 
for teaching mediation have been identified (Cinganotto, 2020). Nevertheless, 
there is still scarce research on the integration of plurilingualism, mediation, and 
technology. For this reason, this paper aims at investigating the ability of teachers 
in Italy and Spain to integrate plurilingualism, mediation, and technology. It also 
aims at comparing the Italian and Spanish results. 


2. Method 


In order to investigate teachers’ awareness of mediation and their ability to integrate 
it with plurilingual and pluricultural competences and with ICTs as well, data were 
collected during two occasions: 


¢ 30/11/2021, Universitat Jaume I, Spain (workshop on mediation for pre- 
and in-service teachers, held within the multiplier event of the TEMPLATE 
project, https://templateplurilingualism.eu/); 


¢ 23/02/2022, Universita degli Studi di Torino, Italy (workshop for pre- 
service teachers proposed within the university course ‘Approaches to 
Language Teaching’). 


In Spain, 52 participants took part in the multiplier event. Most of them (71%) were 
pre-service teachers. The remaining participants were mostly in-service teachers 
(19%). On the other hand, all the 23 Italian participants were pre-service teachers. 
The participants are part of the TEMPLATE project and they are mainly pre-service 
language and/or content and language integrated learning teachers. 


For these workshops, a survey was designed through Google forms in order to 
explore teachers’ awareness of the definition and importance of mediation, and 
their ability to recognise mediation exercises. Indeed, after some questions about 
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metadata (age, occupation, years of experience), the participants were asked to 
define mediation, to explain its importance, and to recognise some mediation 
exercises. 


In order to collect qualitative data, some tasks performed by the participants were 
analysed. In these tasks, they had to integrate mediation, plurilingualism, and 
technology by adapting some mediation exercises to a digital context, through the 
addition of some ICTs and motivating the reason for selecting the tools. Figure | 
below is an example of a task. 


Figure 1. Mediation Task*: Alex and Lara will soon be two new English students 
in your class. You read their description (in English) and you orally 
describe them to your class using Spanish 


ALEX has got 


straight hair and 


dark skin. His eyes are 
SMALL and he has got a 


big smile. 


LARA’s hair is 


and curly. Her 


skin is pale. Her eyes are 
SMALL and she has got 


a big smile. 


The exercise is a cross-linguistic mediation task: the students have to mediate 
orally an English written text, using Spanish. In addition to the verbal input, a 
visual input was provided. The teachers participating in the workshop had to adapt 
this task to a digital context, simply by listing some ICTs that could be included and 
explaining the added value of the inclusion of the tools selected. 


2. Created for this research, using text and images retrieved from https://www.pinterest.es/pin/who-is-who-language-english- 
gradelevel-elemental-school-subject-english-as-a-second-language-es|-mai--452330356328865065/ 
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3. Results and discussion 


The survey results show that most of the Spanish participants (92%) knew the 
definition of mediation, and 96% of them were aware of the necessity to include it 
in the curriculum for educational and social reasons. Nevertheless, most of them 
(63%) did not always recognise mediation exercises. On the other hand, in Italy, a 
smaller number of pre-service teachers (74%) knew the meaning of mediation and 
65% were aware of the importance of including mediation in language courses, 
but 43% of them were not always able to recognise mediation exercises. The 
quantitative data just mentioned are summarised in Table |. 


Table 1. Quantitative results 


Percentage of Percentage of teachers | Percentage of teachers not 
teachers able to aware of the importance | always able to recognise 
define mediation of mediation mediation exercises 

Spain 92% 96% 63% 

Italy 73% 65% 43% 


Analysing the tasks performed by the Spanish participants, it emerged that they 
were able to include ICT tools in existing tasks. They included ICTs in various 
phases of the task and for different reasons: 


¢ for pre/post-task activities (e.g. hangman to revise vocabulary, interactive 
exercises to learn the vocabulary); 


¢ for the task itself (e.g. Canva to prepare a new infographic, Filmora to 
prepare a video, Pixton to create comics, Twitter to summarise the text, 
Grammarly to correct the text, online dictionaries to check vocabulary); 


¢ to facilitate collaboration (videoconference, shared documents). 


Furthermore, they often motivated the use of technology by referring to factors 
not linked to facilitating mediation, such as: motivation, acquisition of digital 
skills, preparation for the job market and academic studies. Italian participants also 
proved to be able to integrate the use of ICTs. Similarly to the Spanish participants, 
they included ICTs: 


¢ for pre/post-task activities (e.g. Mentimeter to elicit vocabulary, YouTube/ 
Quizlet/Kahoot/British council website to present/explain vocabulary, 
collocations, and sentence structure); 
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¢ for the task itself (e.g. Instagram/Prezi to make the text more visual, 
Grammarly to correct the text, online dictionaries, programmes for writing 
effective emails); and 


¢ to facilitate collaboration (videoconference, shared documents, Google 
Drive, Google Classroom). 


When motivating their choices of ICTs, they never referred to mediation but to other 
aspects such as: using tools closer to younger students, increasing amusement in 
learning, interaction and collaboration, and facilitating the production of correct texts. 


4. Conclusions 


In both Italy and Spain pre-service teachers are aware of the definition of mediation. 
Nevertheless, Spanish pre-service teachers are more aware of the importance of 
mediation. In both countries, the participants associate the concept of mediation 
with the educational context and language learning. Even though their theoretical 
knowledge about mediation seems of a good level, when it comes to practise some 
of them encounter difficulties. Indeed, many teachers (63% in Spain, 43% in Italy) 
are not always able to identify mediation exercises. 


Concerning technology, in both countries the attitude is positive, and pre-service 
teachers consider technology as an added value to the task, demonstrating to be 
able to integrate technology in language teaching, but not always focusing on 
the facilitating role that it can have in mediation tasks. Considering the findings 
mentioned above, it is possible to conclude that teachers still need to be trained to 
be able to recognise mediation exercises and to integrate the use of technology in 
mediation tasks, in order to use these tools to facilitate mediation. 
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Intercultural communicative competence development 
through synchronous virtual exchange 
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Abstract. Virtual Exchange (VE) in second language education refers to a 
technology-enabled, interactive, intercultural experience that is designed to increase 
learners’ intercultural communicative competence and performance. In this paper the 
findings of a bilingual (English/Turkish), synchronous video communication project 
among 31 US and Turkish college students are presented. Thematic analysis was 
used in this study. The results show that the VE — to a certain extent and similarly 
to other research studies — achieves the learning objectives of not only increased 
cultural awareness and communicative competence about the target culture and 
language itself but also the growth of factual knowledge about diverse topics that 
the students are interested in learning. 


Keywords: telecollaboration, virtual exchange, intercultural communicative 
competence, less commonly taught languages. 


1. Introduction 


As communicative competence alone is not sufficient for foreign language learning, 
the goal of foreign language education is to develop Intercultural Communicative 
Competence (ICC) skills and cultural awareness to stay relevant in the real-life 
context of the target language (O’Dowd, 2021; Thorne, 2006). Byram (1997, 
as cited in LOpez-Rocha, 2016) suggests that an individual who has developed 
ICC has been defined as someone who builds “relationships while speaking in 
the foreign language; communicates effectively, taking into consideration his own 
and the other person’s viewpoint and needs; mediates interactions between people 
of different backgrounds, and strives to continue developing communicative 
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skills” (p. 107). In recent years, there has been an increasing amount of research 
investigating the role of VE practices on second language learners’ intercultural 
competence (Bailey & Gruber, 2020; Lewis & Qian, 2021). However, there has 
been a scarcity of published research and conference presentations documenting 
the experiences of Less Commonly Taught Languages (LCTLs) students in the 
Computer Assisted Language Learning (CALL) field in general and VE field 
specifically (Ward, 2021). 


This research project has brought together US college students learning Turkish 
and Turkish students learning English to increase their intercultural and linguistic 
competencies. Realizing that less than 2% of all undergraduates in the US have the 
opportunity to study abroad (Feigner, 2017), the researchers have collaboratively 
developed, designed, and implemented this project to bring the global experience 
to Turkish and English language learners, thus developing their linguistic and 
intercultural competence. 


2. Method 


The purpose of this research is twofold: (1) to understand the self-perceived 
effectiveness of participating in VE and (2) to form a basis for future VE research 
in LCTLs. 


A total of 31 US and Turkish college students from various degree programs 
participated in a 50-minute tele-tandem (English/Turkish) synchronous Zoom 
video communication project in the 2021 and 2022 spring semesters. The US 
students were partnered with different Turkish students for each VE session. The 
once a month VE sessions were conducted during the US students’ class sessions 
and hosted three to four pairs of students. After each synchronous VE session, the 
US and Turkish students were asked to respond to an open-ended online survey to 
reflect on their interactions; 104 surveys with open-ended responses were collected 
over two consecutive spring semesters. The students were requested to provide 
examples of areas that VE has contributed to their speaking competence and give 
specific examples in those areas of development. The students were also asked 
to reflect on their overall perceptions of the benefits of participating in a live VE 
session. 


Prior to the VE sessions, the rules of effective virtual communication were clearly 
discussed with the students. The students were provided with globally engaging 
and authentic discussion topics, such as holidays, food culture, college courses, 
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career prospects after college, and the job market. These topics were tied to the 
course curriculum to maintain the rhythm of academic connection. During their 
exchange, the students were also given ample space to talk about topics of mutual 
interest. 


Thematic analysis was used in this study (Creswell, 2014). The students’ open- 
ended statements were initially coded and the identified data were then categorized 
into themes to correspond to the research questions below. 


¢ RQI: What are the common discussion topics among Turkish and 
American college students (aside from the instructor-designated topics) 
during real-time VE? 


¢ RQ2: Do students attribute gains in their language competencies to the 
real-time VE? 


¢ RQ3: In what areas do the students report the real-life VE benefits for their 
language competency development? 


3. Results and discussion 


The results show that the VE, to a certain extent and similar to other research 
studies (Madden & Ashby, 2021), achieves the learning objectives of not only 
increased cultural awareness and competence about the target culture itself, but 
also the growth in factual knowledge about diverse topics that the students are 
interested in learning. 


For RQ1, when the students were asked to report the content of their discussions 
during the real-life VE, besides discussing the pre-set topics that were assigned by 
their instructors, the students also took the initiative to actively engage in various 
topics of mutual interest, such as college, hobbies, places to visit, food, daily life, 
personal life, and post-college life. 


For RQ2, when the students were asked whether they attributed gains in their 
language competencies to the real-life VE, all 31 students responded ‘yes’, despite 
the challenges of meeting across a seven hour time zone difference. 


For RQ3, a chi-square test of independence was performed to examine the relation 
between student target language and Theme | (self-confidence in speaking, 
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Table 1). The relation between these variables was significant: X° (1, N=97)=4.52, 
p=.033. The inference is that Turkish students were not likely to report improved 
self-confidence in speaking English as a target language. Theme | accounts for 
20.20% of the total coded comments. 


A chi-square test of independence was performed to examine the relation between 
student target language and Theme 2, pronunciation improvement (Table |). The 
relation between these variables was significant: X? (1, N=97)=5.02, p=.025. The 
inference is that Turkish students were likely to report improved pronunciation in 
speaking English as a target language, and the American students were not likely to 
report improved pronunciation in speaking Turkish. Theme 2 accounts for 23.33% 
of the total coded comments. 


A chi-square test of independence was performed to examine the relation 
between student target language and Themes 3 (improvement in vocabulary) and 
4 (speaking and fluency improvement, Table |). The relationship between student 
target language and the variables of Themes 3 and 4 was not significant: X? (1, 
N=97)=0.31, p=.580 and _X? (1, N=97)=2.69, p=.101, respectively. There was no 
relationship between student target language and Themes 3 and 4. Themes 3 and 
4 account for 20.71% and 22.73% of the total coded comments, respectively. 


A chi-square test of independence was performed to examine the relation between 
student target language and Theme 5 (grammar improvement, Table 1). The relation 
between these variables was significant: X? (1, N=97)=7.78, p=.005. The inference 
is that Turkish and US students were not likely to report improved grammar in 
their respective target languages. Theme 5 accounts for 13.3% of the total coded 
comments. 


Table 1. Self-reported gains in their language competencies to the real-life VE 


Value df Asymptotic Significance 
(two-sided) 

Theme 1: 4.522 1 .033 
Self-confidence in speaking 
Theme 2: 5.020 1 025 
Pronunciation improvement 
Theme 3: 306 1 580 
Vocabulary improvement 
Theme 4: 2.690 1 101 
Speaking and fluency improvement 
Theme 5: 7.7719 1 005 
Grammar improvement 
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4. Conclusions 


In this study, the self-perceived effectiveness of participating in real-time tele- 
tandem VE among college students from two countries with a seven-hour time 
difference was investigated, and an attempt was made to form a basis for future VE 
research in LCTLs that might help address the neglect of this topic in the literature 
and conference presentations. As the results indicate, the students proactively took 
the initiative to explore various themes and took charge of their own learning. 
The students who participated in VE attributed gains in their language competency 
to the real-life VE and reported language competency gains (to a certain extent) 
in five areas (self-confidence in speaking, pronunciation, vocabulary, fluency, and 
grammar). Future research could include VE partnerships among other institutions 
and languages and increase the number of students and the amount of time allocated 
for the real-time exchange. 
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ICALL ecosystems: making ICALL’s intelligence 
both accessible and understandable 


Jasper Degraeuwe'! and Patrick Goethals? 


Abstract. This paper presents a reflection on the design of an Intelligent Computer- 
Assisted Language Learning (ICALL) ‘ecosystem’, integrated into an online 
learning environment for Spanish as a Foreign Language (SFL). The innovative 
dimension of the ecosystem lies in its triple focus: apart from enabling users to create 
and use intelligent language learning materials, it also tracks their activities in the 
environment and provides them insights (e.g. through knowledge clips) into Natural 
Language Processing (NLP), the source of ICALL’s ‘intelligence’. The reflective 
analysis is carried out by means of a case study with 32 SFL students, who work with 
the ecosystem in a blended writing course focused on vocabulary learning, lexical 
ambiguity, and Word Sense Disambiguation (WSD). Students’ attitudes towards 
engaging in the ICALL ecosystem are gauged through a questionnaire, which 
revealed a statistically significant positive change in attitude after having completed 
the course. However, the results also show that enhanced insights into NLP and 
increased confidence in the computer as a learning assistant do not necessarily go 


hand in hand with an increased curiosity and a better user experience. 


Keywords: ICALL, NLP, reflective analysis, user attitudes. 


1. Introduction 


With applications such as example sentence selection systems (Pilan, Volodina, & 
Borin, 2016) and exercise generation tools (Zanetti, Volodina, & Graén, 2021), the 
implementation of ICALL in language learning courses can be a valuable addition 
to the arsenal of teaching methods, for example as a complement to on-campus 
vocabulary learning activities (Ruiz, Rebuschat, & Meurers, 2021). Nevertheless, 
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using ICALL also comes with its limitations. Recognising lexically ambiguous 
items and distinguishing between their senses is one of those pending issues (Pilan 
et al., 2016), as the NLP-driven technique of WSD is not yet integrated in corpus 
query tools or in the development of computer-readable resources for didactic 
purposes, such as graded word lists. Additionally, it should be taken into account 
that, in order to get the most out of ICALL environments, end users (which may 
refer to students, teachers, or even textbook designers) should possess certain 
‘technological metaskills’ (Schweinberger, 2021) that allow them to decide which 
queries are relevant and feasible to perform. In this study, we carry out a case 
study to analyse how both above mentioned aspects can be tackled by designing 
an ICALL ecosystem. 


2. Method 


2.1. Ecosystem design 


Conceptually, the ecosystem should enable users to generate and use customised 
learning materials (Aspect_1), as well as help them gain technological metaskills 
by stimulating their curiosity and promoting their autonomy (Aspect_2). In the 
meantime, all activities of users who give their informed consent end up in a 
structured database, which can then be used for improving the NLP-driven methods 
integrated into the environment (Aspect_3). 


2.2. Case study design 


The participants of the study are 32 SFL students enrolled in a third bachelor (B2+ 
level) Spanish writing course at Ghent University in Belgium. During the course, 
they work with the online learning platform of the Spanish Corpus Annotation 
Project? (SCAP; Goethals, 2018), which includes: 


* a section on corpus consultation, in which users can perform targeted 
queries based on part-of-speech and lemma information and generate 
lemma lists with frequency and keyness values; 


* asection on vocabulary learning, in which users can automatically generate 
customised vocabulary lists/glossaries and fill-the-gap exercises; and 


3. Publicly available version of the platform accessible through scap.ugent.be. Demo video of the in-house version available at 
youtube.com/watch?v=RFalWEEZcVM 
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¢ acollaborative section for research purposes. 


Part of the course consists of completing a blended vocabulary learning module, 
which encompasses two on-campus classes and an online module on lexical 
ambiguity. During the classes, the students learn to use the corpus consultation 
and vocabulary learning functionalities of the SCAP platform (Aspect_1). For 
the online module, they consider lexical ambiguity from the perspective of the 
computer by watching knowledge clips* and develop their own WSD models 
by making interactive exercises on lexically ambiguous vocabulary items in the 
collaborative section of the platform’ (Aspect_2). The responses to those exercises 
(see Figure | for an example) are collected in a database and used to develop the 
actual WSD method integrated into the environment (Aspect_3). 


Figure 1. Example of interactive exercise on lexical ambiguity 


Ejercicio de desambiguacién — Parte 2 


En la segunda parte del ejercicio, vas a llevar el desarrollo del sistema de WSD un paso mas alla. Abajo te presentamos las 10 frases en los corpus de SCAP que son las mas dificiles 
para predecir para el sistema en base a las 2 frases prototipicas clasificadas por ti en la primera parte del ejercicio. El objetivo es que ayudes al ordenador a resolver estos casos 
dificiles, para ver si puedes llegar a una mejor versién del modelo de WSD. Para ello, selecciona otra vez el significado correcto en el ejercicio abajo, o indica ‘Otro / 7 sino estas 
segur@ del significado al que pertenece la frase. Pero ten cuidado, esta vez el ejercicio no se corregira, es tu responsabilidad pensar bien y ofrecer al sistema frases Clasificadas 
correctamente. Al dar en el botén ‘Mostrar grafico’, se mostrara un nuevo grafico, en que se han afiadido los vectores de las frases que acabas de clasificar. 


frase moneda simbolo, Otro/ Comentario 
extranjera eslogan ? 


1) Y debajo habian incluido la divisa familiar : Vivitur ingenio , caetera mortis erunt 


2.3. Questionnaire 


To gain insights into the potential of the ecosystem, the students are administered 
an adapted version of the A-CALL questionnaire (Attitude towards CALL; 
Vandewaetere & Desmet, 2009) before and after completing the vocabulary 
learning module. The questionnaire contains fifteen eight-point Likert scale 
questions, each of them representing a specific attitude towards ICALL (see 
Table 1). 


3. Results 


Table | reports the mean scores and standard deviation (SD) values of the 
15 questions (four students failed to complete the module, which explains the 


4. Complete video available at youtu.be/-evS6uEpIkA 
5. Demo video available at youtube.com/watch?v=OjPWKEpYiAA 
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different value for n). A paired samples t-test revealed a statistically significant 
(p=0.01) difference, meaning that engaging with the ecosystem positively affected 
students’ attitudes towards ICALL (as represented by the 15 average scores). 
However, the results also show that enhanced insights into NLP (Questions 1, 3, 5, 
and 9) and increased confidence in the computer as a vocabulary learning assistant 
(4 and 6) do not necessarily go hand in hand with an increased curiosity (2) and a 


better user experience (11). 


Table 1. 


reversed 


Questionnaire results, with scores on questions marked with “ being 


Question 


Pre (n=32) 


Post (n=28) 


Mean 


SD 


Mean 


SD 


The computer is able to analyse the 
grammatical characteristics of words, and 
link words to their corresponding part-of- 
speech (noun, verb, adjective, etc.). 


5.12 


1.62 


7.18 


0.86 


I am interested in knowing more 
about the technology which enables 
computers to automatically create 
vocabulary exercises and resources. 


4.16 


2.2 


3.89 


1.87 


The computer only sees sequences of 
letters which are combined into words, it 
is not able to see meanings and concepts 
behind these sequences of letters. 


4.91 


1.51 


5.96 


1.48 


I have confidence in computer-created 
vocabulary exercises and tests. 


4.69 


1.31 


5.86 


1.3 


If I introduce a large collection of texts 
on a certain domain into a specific 
application, I think that this application 
will be able to return a keyword list with 
the most typical words for the domain. 


5.56 


1.37 


6.71 


1.05 


The computer is able to generate 
vocabulary exercises and resources 
tailored to my proficiency level. 


5.34 


1.21 


6.68 


1.22 


The teacher’s attitude and enthusiasm 
towards and knowledge of computer-assisted 
vocabulary learning determine to a large 
extent my attitude towards using computers 
for vocabulary learning purposes.” 


3.22 


1.77 


3.54 


2.12 


Computer-assisted vocabulary learning offers 


more flexibility to learning vocabulary in Spanish. 


5.53 


1.5 


5.61 


1.89 


The computer is able to analyse the 
syntactic structure of sentences, and 
assign the correct syntactic function 
(subject, direct object, etc.) to words. 


4.53 


1.27 


5.61 


1.47 
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10 Computer-assisted vocabulary learning 4.28 1.49 = 4.82 1.49 
is as valuable as traditional methods for 
vocabulary learning in Spanish. 


11 I (would) like to learn Spanish vocabulary 5.28 | 2.1 4.89 1.91 
with the help of the computer. 
12 I find it easier to accept an error committed 4.06 1.78 (4.07 | 2.02 


by a language teacher, than an error 
committed by the computer. 

13 People who learn Spanish vocabulary 6 1.93 6.29 1.41 
through computer-assisted learning methods 
are less proficient in Spanish than people 
who learn Spanish vocabulary through 
traditional paper-and-pencil methods. 


14 Computer-assisted vocabulary learning is 6.09 1.47 6.07 1.74 
a valuable extension of traditional learning 
methods for vocabulary learning in Spanish. 


15 | Vocabulary exercises and resources 3.34 (2.06 | 3.18 1.93 
created automatically by an application 
cannot contain errors." 


481 1.64 5.36 | 1.58 


4. Discussion and conclusions 


In this paper we reflected upon the design of an ICALL ecosystem using a specific 
questionnaire as our survey instrument. Working with the ecosystem significantly 
improved students’ attitudes towards ICALL, but this did not automatically mean 
that they also enjoyed working with the computer more, or that the ecosystem 
sparked their interest in learning even more about language technology. In other 
words, these preliminary findings highlight the area of tension between what 
students consider to be the value, quality, and/or potential of learning methods, and 
the user experience these methods provide. To analyse this phenomenon in more 
detail, we will organise a follow-up case study in the 2022-2023 academic year, 
in which we will evaluate a revised version of the ecosystem based on this year’s 
results. 
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Does participating in a telecollaborative project 
foster the acquisition of apologies? Insights from 
the English for specific purposes context 


Sofia Di Sarno-Garcia! 


Abstract. The aim of the study was to investigate whether telecollaboration is a 
suitable environment for the acquisition of the speech act of apologies. Participants 
were aerospace engineering students from the Universitat Politécnica de Valencia 
(Spain) who performed six open role-plays to elicit apologies. The Control Group 
(CG, n=17) carried out the task in pairs with their Spanish classmates in a Face-To- 
Face (FTF) setting, while the Experimental Group (EG, n=7) conducted the task 
with first language (L1) or highly proficient speakers of English from the University 
of Bath (UK) through synchronous Zoom sessions. The results of the descriptive 
analysis revealed a higher tendency of improvement in the EG, which also used 
a higher number of strategies compared to the CG. Findings from the quantitative 
analysis carried out through an Eta coefficient revealed a significant correlation 
(r=.71) between the number of strategies used and the modality where they were 
performed. 


Keywords: synchronous computer-mediated communication, telecollaboration, 
cyberpragmatics, speech acts, apologies, English for specific purposes. 


1. Introduction 


Since Yus (2011) coined the term cyberpragmatics to describe “the online focus 
of pragmatics” (Orsini-Jones & Lee, 2018, p. 26), the interest in analysing the 
pragmatic implications of online encounters has increased, although the field 
of second language (L2) pragmatics in Computer-Assisted Language Learning 
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(CALL) is still under-researched (Blyth & Sykes, 2020). Nevertheless, 
cyberpragmatics plays a crucial role in telecollaboration, and for that reason 
students in higher education should be trained on that to become global citizens 
(Orsini-Jones & Lee, 2018). Furthermore, telecollaboration can be beneficial for 
the teaching and learning of L2 pragmatics because it fosters learners’ exposure 
to L2 pragmatic features (Belz, 2007; Gonzalez-Lloret, 2021; Rafieyan et al., 
2014). 


Following the recommendation made by Blyth and Sykes (2020), this paper focuses 
on L2 pragmatics and digitally-mediated interaction through video conference 
sessions. Specifically, we focused on the speech act of apologies, which are a 
characteristic of English-speaking societies (Leech, 2014). In fact, according to 
Halenko (2021), 


“research shows producing pragmatically appropriate language in a British 
context may be particularly problematic for international students from 
positive politeness cultures (Brown & Levinson, 1987) such as Spain” 
(p. 6). 


Moreover, it should be highlighted that, while grammatical errors can be tolerated, 
pragmatic mistakes can cause cultural misunderstandings (Gonzalez-Lloret, 2021), 
and can take place despite the high linguistic proficiency of the learner (Blum- 
Kulka & Olshtain, 1984). The next section will explore the methodology followed 
to conduct the study. 


2. Method 


This paper focuses on the improvement of Spanish students’ use of apologies. For 
that reason, we will compare the results obtained from a CG and an EG. 


Participants of the CG were 17 Spanish-speaking students from the Universitat 
Politécnica de Valéncia, while those in the EG were seven Spanish students from 
the same university who participated in a telecollaboration project with highly 
proficient or L1 speakers of English from the University of Bath. They were enrolled 
in an optional third-year B2 level (CEFR) English for specific purposes subject in 
the aerospace engineering degree. All participants completed a questionnaire as a 
pre-test before receiving the treatment, which consisted of explicit instruction on 
pragmatics and apology performance by means of a PowerPoint presentation and 
audiovisual input. Afterwards, participants were expected to carry out six open 
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role-plays (one per week) whose main aim was to elicit apologies. The difference 
between the CG and the EG lies in the fact that the former carried out the task with 
their Spanish classmates in a traditional FTF setting, while the latter performed 
the role-plays in synchronous video conference sessions with their partners from 
Bath. At the end of the project, students from both groups completed the same 
questionnaire as a post-test. The survey was composed of ten multiple choice 
questions aimed at gathering information about the participants’ use of apologies 
before and after the treatment. 


The role-plays were recorded and transcribed. The strategies used to apologise 
were coded following a taxonomy based on Blum-Kulka and Olshtain (1984), 
Leech (2014), and Martinez-Flor (2016), and its frequency was calculated by 
means of a quantitative content analysis. Furthermore, the descriptive statistics of 
the responses from the pre- and post-test for each group were compared. Finally, 
the Eta coefficient was calculated to see if there was any relation between the 
number of strategies used by the members of each group and the modality where 
they were performed (i.e. FTF or telecollaboration). 


3. Results and discussion 


A total number of 81 role-plays were transcribed and analysed. Table | illustrates 
the results obtained from the classification of apologies strategies. An extra 
category was added for those strategies which did not belong to any of the 
established categories, but which we believed were the result of participants’ L1 
transfer. In terms of frequency, the EG used a slightly wider range of strategies 
compared to the CG. In fact, even though the EG was smaller than the CG, it 
used almost the same number of strategies. The main categories used by the CG 
are expressions of the Speaker’s (S) regret such as J am sorry, explanations of 
why the fault occurred, and using a performative utterance such as / apologise. 
Conversely, apart from those strategies, the EG used much more apology 
intensifications such as truly (sorry), showing concern for the Hearer (H), and 
promises of forbearance. 


Table 2 summarises the findings from the descriptive analysis of the CG. The mean 
(M) revealed that there was a slight improvement only in Items 2, 3, 8, and 10. 
Conversely, in the case of the EG the improvement was observed in the M of 
Items 1, 3, 6, 7, 8, and 9, while Items 4, 5, and 10 showed no variation (Table 3). 
A regression was observed only in Item 2, while different items of the CG showed 
a slight regression. 
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Table 1. Frequency of strategies used to apologise 


Strategy Type CG EG 
n % n % 
Head act Expression of S’s regret 74 38.54 49 26.77 
Asking H’s pardon 1 0.52 1/3 1.64 
or forgiveness 
Using a performative utterance | 11 5.73 |5 2.73 
Expression of Explicit self-blame 7 3.65 |4 2.19 
responsibility Denial of fault 1 0.52 |1 0.55 
Explanation of why 41 21.35 28 15.30 
the fault occurred 
Offer of repair 15 7.81 17 9.29 
Promise of forbearance 8 4.17 16 8.74 
Apology intensification | Concern for the H 4 2.08 | 12 6.56 
Intensifier/modifier 26 13.54 38 20.77 
LI transfer 4 2.08 10 5.46 
TOTAL N of 192 184 
strategies used 
TOTAL N of role-plays 50 31 


Table 2. Descriptive statistics of the CG 


M SD 
Item | Pre-test 2.762 0.539 
Post-test 2.647 0.786 
Item 2 Pre-test 2.810 0.512 
Post-test 3.000 0.000 
Ttem 3 Pre-test 2.048 1.024 
Post-test 2.294 0.985 
Ttem 4 Pre-test 2.810 0.402 
Post-test 2.824 0.393 
Ttem 5 Pre-test 2.905 0.436 
Post-test 2.882 0.485 
Ttem 6 Pre-test 2.524 0.750 
Post-test 2.353 0.931 
Item 7 Pre-test 2.952 0.218 
Post-test 2.824 0.529 
Item 8 Pre-test 2.667 0.577 
Post-test 2.706 0.588 
Ttem 9 Pre-test 3.000 0.000 
Post-test 2.941 0.243 
Item 10 Pre-test 1.952 0.740 
Post-test 2.235 0.664 
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Table 3. Descriptive statistics of the EG 


M SD 
Ttem | Pre-test 2.857 0.378 
Post-test 3.000 0.000 
Ttem 2 Pre-test 3.000 0.000 
Post-test 2.714 0.756 
Item 3 Pre-test 2.143 1.069 
Post-test 2.429 0.976 
Ttem 4 Pre-test 3.000 0.000 
Post-test 3.000 0.000 
Ttem 5 Pre-test 3.000 0.000 
Post-test 3.000 0.000 
Item 6 Pre-test 2.143 0.690 
Post-test 2.571 0.787 
Item 7 Pre-test 2.857 0.378 
Post-test 3.000 0.000 
Item 8 Pre-test 2.857 0.378 
Post-test 3.000 0.000 
Item 9 Pre-test 2.857 0.378 
Post-test 3.000 0.000 
Item 10 Pre-test 2.000 0.577 
Post-test 2.000 1.000 


Finally, the results of the Eta coefficient showed a statistically significant 
correlation between the number of strategies used to apologise by each student and 
the work modality as r =.71, which means that those students who participated in 
the telecollaboration used a higher number of strategies to apologise. 


4. Conclusions 


The results of this study shed light on the effect that telecollaboration can have 
on the acquisition of the speech act of apologies. Although the main limitation is 
the small sample size, they showed that there is a relation between the number 
of strategies used to apologise during role-plays and participating or not in a 
telecollaboration. Moreover, the findings from the pre- and post-test revealed 
a higher tendency of improvement in the case of the EG. The study will be 
replicated to corroborate these results. In the light of the results obtained, future 
research should investigate the effects of telecollaboration on the acquisition of 
speech acts. 
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Abstract. This study aimed to investigate how the enforced online language 
teaching operated in a limited resource context, i.e. Egypt. Based on the Technology 
Acceptance Model (TAM), a nation-wide survey examined how 258 language 
teachers perceived the sudden move to remote online teaching and whether such 
perceptions had an impact on their readiness and intention to use this mode during 
and after the pandemic. Confirmatory factor analysis was used and revealed that 
four factors constitute teachers’ perceptions. Results showed that participants have 
positive perceptions of online teaching, which consequently indicate that they 
were ready to use online language teaching even after switching back to face-to- 
face. Demographic data had no statistically significant effect on the participants’ 
perceptions of enforced online teaching. Teachers identified training needs that 
show readiness and intention to engage more with this mode of teaching. 


Keywords: remote online teaching, EFL, TAM, Egypt. 


ale Introduction 


Published research prior to COVID on key challenges facing education in Egypt 
includes issues of over-centralized control, exam-orientation, and entrenchment of 
social inequalities (Loveluck, 2012). Evidence suggests the challenges are more 
severe for vulnerable groups including learners with poor socio-economics (Ersado 
& Gignoux, 2014), learners with disabilities, refugees, and asylum seekers. The 
swift move to remote online teaching during the pandemic was unforeseen and 
unprecedented. Additionally, training programs do not address the needs of online 
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teachers, which Moeini (2008) describes as ‘the missing part of online training 
programmes’. Studies in the research context focused on university levels (e.g. 
El-Sayad, Saad, & Thurasamy, 2021; Esawe, Esawe, & Esawe, 2022), but less 
attention was given to school teachers. The main research questions for this study 
were as follows. 


¢ To what extent has the remote online teaching approach during the 
pandemic shaped or re-shaped teachers’ readiness to use technology for 
language teaching? 


¢ To what extent have the demographic variables influenced teachers’ 
experiences during the pandemic, their needs, and intentions to use 
technological tools in the future? 


2. Theoretical background 


The use of online teaching tools in English as a Foreign Language (EFL) contexts 
has been the subject of research for a few decades focusing on issues such as 
perceptions of technology use (e.g. Cheng, 2007), interactive behaviors (e.g. Liao 
& Lin, 2011), and evaluating online teaching/learning (e.g. Chan, Chow, & Jia, 
2005; Novo-Corti, Varela-Candamio, & Ramil-Diaz, 2003) among others. TAM has 
been a general framework for many studies. TAM was introduced by Davis (1986, 
1989, 1993) more than four decades ago, but it still forms a significant element of 
current studies. Charness and Boot (2016) note TAM’s two primary factors affecting 
an individual’s intention to use new technology: Perceived Usefulness (PU) and 
Perceived Ease of Use (PEU). Davis (1986, 1989, 1993) explains these as the 
individual’s own likelihood that using a technological tool can improve tasks with 
relative ease. Attitudes are related to the individual’s appraisal of the need to use 
certain technological tools to fulfill his/her task. Behavioral intentions are concerned 
with the possibility of using the tool involved by this individual in the future. 


3. Method 


3.1. Instruments 


A questionnaire of 24 Likert-type items was designed to collect data about EFL 
teachers’ perceptions of the enforced online teaching and the variables shaping/ 
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reshaping them. To validate the questionnaire, four experts examined the items 
for content validity and confirmatory factor analysis was used to assess its 
construct validity. Cronbach’s alpha analysis was computed to examine reliability. 
Translation and back-translation procedures were implemented followed by a pilot 
on six teachers to investigate item clarity and appropriateness, instructions, and 


format. 


3.2. Participants and procedures 


The questionnaire was delivered via Qualtrics in December 2021, which resulted 
in 258 responses. Respondents came from state, private, and international 
institutions representing 16 out of the 27 geographical regions in the country. 
Age, educational qualifications, and teaching experience are shown in Table | 


below. 


Table 1. Demographics 


Variable n % Variable n % 

Age Qualifications 

<25 23 8.9 BA 112 43.4 

25-30 37 14.3 Diploma 91 35.3 

31-35 52 20.2 MA 63 14 

36-40 47 18.2 PhD 9 3:5 

41-45 45 17.4 Other 8 3.1 

46-50 33 12.8 Missing 2 0.8 

51+ 11 4.3 

Missing 10 3.9 

School Type Teaching Experience 

Public 68 26.4 <2 7 27, 

Experimental | 54 20.9 2-5 36 14 

Azhariate 10 3.9 6-10 54 20.9 

Private 56 21.7 11-15 55 21.3 

International | 60 23.3 61-20 35 13.6 

Missing 10 3.9 20+ 68 26.4 
Missing 3 1.2 

Teaching Stage 

Primary 102 39.5 

Intermediate/ | 64 24.8 

Preparatory 

High/ 70 27 

Secondary 

Missing 22 8.5 
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3.3. Analysis 


Three statistical techniques were used, namely Confirmatory Factor Analysis 
(CFA), and descriptive and inferential statistics. CFA was used to understand 
the interrelationships among the questionnaire variables and their underlying 
dimension(s). Descriptive statistics were conducted to the participants’ responses. 
Inferential statistics (i.e. Kruskal-Wallis and One-way ANOVA, followed by 
the Scheffe post hoc-test) were computed to scrutinize the impact of biodata on 
participants’ perceptions of and readiness for online teaching. 


4. Results 


4.1. Factors influencing perceptions 


The CFA suggested four dimensions constituting the construct of teachers’ 
perceptions of enforced online teaching which are (1) Attitude, Perceived Impact 
and Future Intention (APIFI), (2) PEU, (3) Perceived Challenges (PC), and (4) 
PU. Cronbach’s alphas were 0.93, 0.77, 0.77, and 0.58 for Factors 1, 2, 3, and 4 
respectively. 


4.2. Teachers’ perceptions 

Item descriptive statistical analyses (frequencies (F), means (M), and standard 
deviations (SD)) revealed that the respondents had positive perceptions of enforced 
online teaching. Consequently, this reflects their readiness to endorse online 
teaching and their intention to use it in the future. 


Table 2 illustrates the descriptive statistics of participants’ responses. 


Table 2. Descriptive statistics 


Factors Number | Responses 
or homs (Strongly) Agree — Can’t Decide — (Strongly) Disagree 
M SD 
APIFI | 13 3.05 — 4.02 0.858 < SD = 1.202 
PEU 5 3.51 — 3.94 0.881 < SD = 1.092 
PC 5 3.10 —3.35 1.070 < SD = 1.269 
PU 2 3.06 — 3.39 1.136 < SD => 1.154 
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4.3. Variables shaping/reshaping perceptions 

Inferential statistics showed there was no statistically significant difference 
among participants’ mean scores on the overall questionnaire and across its 
factors according to their age, school type, teaching stage, teacher qualifications, 
and teaching experience. The only significant difference found was related to 
Factor 2 (PEU), with primary stage teachers’ increased perception of ease of online 
teaching use (t=0.001, p=0.000). Table 3 shows the factors of teachers’ perceptions, 
variables tested, statistical tests used, values obtained from each test, and statistical 
significance. 


Table 3. Kruskal-Wallis and One-way ANOVAs related to the relationships 
between teachers’ demographics and perceptions 
Variables and Statistical Tests 
~ | Age School Type | Teaching Stage | Qualifications | Experience 
= Kruskal- ANOVA and |ANOVAand |ANOVAand_ | ANOVA and 
| Wallis Post-hoc-test | Post-hoc-test | Post-hoc-test | Post-hoc-test 
(H) (Scheffe) (Scheffe) (Scheffe) (Scheffe) 
(F-ratio) (F-ratio) 
Fl | 0.173 (df=2) | 0.534 0.131 0.081 0.084 
F2 | 0.282 (df=2) | 0.443 0.001 [Pr]* 0.089 0.056 
F3 | 0.087 (df=2) | 0.448 0.834 0.519 0.471 
F4 | 0.193 (df=2) | 0.098 1.340 0.263 0.641 
Notes | *Significant at 0.05 
Pr=Primary Stage 


The questionnaire open-ended responses revealed that involvement in enforced 
online teaching led participants to reflect on their practice and identify pedagogical 
training needs which could not have been identified otherwise. Some of these 
were online classroom management, online testing, online interaction and 
engagement, supporting learning disabilities, and materials accessibility. Such 
results support their claims for readiness and intention to engage more with this 
mode of teaching in the future. 


5. Discussion and conclusion 

This study investigated EFL teachers’ perceptions, readiness, and intention to 
integrate online teaching during and after COVID. Findings revealed that teachers 
had positive perceptions of online teaching, which indicated they were ready 
to endorse this type of practice in the future. Demographics had no impact on 
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perceptions of online teaching. The need to deliver education to students urged 
teachers to use their limited resources during the pandemic despite lack of adequate 
training. Data suggested that primary teachers had more positive perceptions in 
relation to ease of/intention to use online teaching. This could be attributed to 
the educational reform of primary education adopted by the government which 
requires primary material to be delivered online by 2024. Therefore, enforced 
online teaching helped teachers to identify a partial substitute for face-to-face 
teaching and understand their training needs. 
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L2 Chinese language teachers’ cross-cultural 
adaptation in teaching online courses using 
videoconferencing tools in a foreign country 
during the COVID-19 pandemic: a case study 


Yue Fan', Li Cheng’, and Zhu Zhu* 


Abstract. This case study is based on teachers’ personal observation of students 
of Oberlin College, Ohio, US, and students’ feedback, and addresses cross-cultural 
communication of Chinese as a foreign (L2) language teacher, who is also the first 
author of this article. The study was conducted during the global pandemic; L2 
language teaching methods in Oberlin College had to be shifted from face-to-face 
lectures to online teaching using videoconferencing tools, particularly Zoom. It was 
used not only as the online class platform, but also an additional live communication 
tool in other activities. The case study presented in this article was conducted mainly 
through observations in daily classes before and after the pandemic. The results 
suggest that reserved personality and inadequate cultural contact are factors of 
accultured difficulties for L2 Chinese language teachers when working in the US. 
This paper proposes solutions for preparations for a cross-cultural adaptability for 
Chinese language teachers teaching L2 Chinese abroad, especially in conditions like 


using videoconferencing tools in online teaching classes. 


Keywords: TCSOL, cross-cultural communication, L2 Chinese language teaching, 
L2 Chinese teachers abroad. 
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“7 Introduction 


Teaching Chinese to Speakers of Other Languages (TCSOL) has gradually become 
an important field of L2 language teaching after China’s Reform and Open Policy. 
Since then, some Chinese universities started a TCSOL study major and recruited 
students to become L2 Chinese language teachers. Most studies of TCSOL 
teachers have focused on their teaching pedagogy of language itself, and there are 
relatively few discussions on their cultural adaptation when working in a foreign 
country. Some scholars indicate that teaching ability of teachers of L2 Chinese 
abroad includes at least three aspects: professional knowledge, working ability, and 
rich experience in teaching practice (Lv, 1989). The working ability includes the 
communicative ability, which mainly refers to the proper behavior and language in 
formal occasions and classroom teaching. It also includes a strong communicative 
ability in organizing, adjusting, and dealing with emergency situations. 


When reviewing this from a cross-cultural perspective, the communicative ability 
could be an independent and important aspect. It may not only be an attachment 
of L2 language teachers’ teaching ability, but also the embodiment of the teachers’ 
cross-cultural communication skills. One famous theory is the u-curve theory of 
cultural adjustment, which is a framework that focuses on how people adjust to 
being in a new culture (Lysgaand, 1955). The u-curve theory of adjustment consists 
of four stages of adjustment (honeymoon, culture shock, adjustment, and mastery). 
We have an assumption that the influence of L2 language teachers’ current stage is 
as vital as their teaching approach. Therefore, this paper focuses on the obstacles 
faced by L2 Chinese language teachers in cross-cultural teaching. 


The two research questions are as follows. 


(1) What are the pros and cons online teaching classes have brought to L2 
Chinese language teachers working in foreign countries? 


(2) How can we help novice L2 language teachers to improve the accultures 
experience when using videoconferencing tools in online teaching classes? 


2. Design of the case study 


Observation and experience: The first author was one Chinese language teacher 
of L2 Chinese in Oberlin College in the US. The observation was taken in daily 
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teaching for half a month before turning into online class, and also another 
half-month’s observation after having online class via Zoom. What was mainly 
observed included daily activities in L2 Chinese classes: daily lesson and Chinese 
table twice a week (a lunch time when language teachers and students gathered and 
the students were asked to chat in Chinese, or at least act out their newly learned 
dialogue). The observation experience was recorded by the teacher in teaching 
journals. The teacher observed their own actual experience in teaching and daily 
life, noted down different cross-cultural situations before and after the COVID-19 
pandemic, and experienced the practical effects of the online conferencing tool 
Zoom. 


Evaluations and interviews: The students’ evaluation was a form which was 
released to students at the end of one semester. It contains study effect self-rating 
and one commenting question at the end. With the help of students’ evaluations in 
May 2020, students’ comments were collected, and the students agreed to release 
them to teachers. Also there were two students who were consent to talk about 
their feelings during the Chinese table. Ongoing analysis was employed to analyse 
the data. The researcher analyzed the data by repeatedly reading and recalling the 
recorded experience. 


3. Results and discussion 


The results showed that online L2 Chinese language teaching has both pros and 
cons for L2 Chinese teachers’ cross-cultural experiences. 


From self-observation, the interactions between the teachers and the students 
before the pandemic was not limited to classrooms, but after the pandemic, 
the communication space became sharply compressed to the Internet only. 
Although digital face time can still be guaranteed with the help of Zoom or other 
videoconferencing tools, students have lost most of the opportunities to directly 
contact and interact with their teachers. Teachers would no longer have other 
cultural communication scenarios except for the online class time. Following 
Lysgaand’s (1955) u-shape curve theory of acculturation, online teaching greatly 
extends the current stage of acculturation for teachers. 


However, the videoconferencing tool Zoom also offered a great help to language 
teaching. The monitors hinder face-to-face communication between teachers and 
students, but indeed expand the space compared with the traditional classroom. 
For example, Oberlin College has the Chinese table activity. Before the pandemic, 
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students were asked to take their lunch boxes to the venue to eat and chat in 
Chinese, but after teaching online, teachers could show a simple Chinese dish in 
their kitchen, and teaching assistants could display necessary Chinese vocabulary 
in the chat box. By the end of the activity, some of the students were able to briefly 
introduce how to make a Chinese dish by using the grammar and new vocabulary. 
Such examples show a great benefit to a multimedia assisted language study as 
well as to the process of acculturation. 


Students also mentioned that in previous face-to-face teaching, in addition to 
speaking, eyes and gestures were used by teachers intentionally or unintentionally 
to convey additional information, which added to students’ understanding and 
attractions. In the screen time, however, only the upper body of the teacher could 
be seen from the screen, and less body language would be perceived by students 
when the teacher’s image was restricted into a small window of videoconferencing 
tools. According to students’ evaluations in May 2020, descriptions like 
enthusiastic, motivated, humorous, and lively were mentioned in the commenting, 
but appeared as the comparative good things of their native US college lecturers’ 
online performance. Students addressed these words to indicate that without face- 
to-face or eye-to-eye contact, L2 language teachers needed more passion to create 
a lively atmosphere. A lively atmosphere, in students’ opinions, turned out to be an 
important attraction during online class. 


4. Conclusions 


This case study showed that the cross-cultural experience of L2 Chinese language 
teachers in foreign countries has actually been weakened by online distance 
teaching to a certain extent, but a new and positive perspective should provide 
insights about the benefits of using videoconferencing tools in online teaching 
classes. The experience of the first author and her students showed some of the 
possibilities, such as using Zoom in the Chinese table activity, to break the space 
limit of a traditional classroom and shift the study environment to anywhere that 
could motivate students’ expressions. 


Moreover, it is a useful way to utilize videoconferencing tools to create a sense 
of community in order to maintain a good bonding of teachers and students. 
Another possible approach is that, when teaching the module ‘positions of objects’, 
if consent, teachers and students could show their own room and talk about the 
position of anything in it. With the help of Zoom, the students could learn more 
about each other, find more topics for discussion, and improve their own interest in 
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learning a foreign language. This may require teachers’ recognition of local culture 
and an open, inclusive mentality. This kind of literacy would be acquired from 
teacher training preparation before leaving their own country to teach in a foreign 
country. Therefore, TCSOL teacher training institutions should pay more attention 
to personality adaptation by creating typical cultural scenes and avoiding potential 
conflicts. To be more specific, preparing the teacher to use their bodily language 
fully on screen when teaching online using videoconferencing tools. Training 
institutions could organize online teaching training modules where teachers could 
review their screen recording in order to experience the teaching during an online 
class scenario before the actual practice. 
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Abstract. This study examines the relationship between English language proficiency 
outcomes and the use of a Computer-Assisted Language Learning (CALL) tool that 
is designed to strengthen the academic English and oral proficiency skills of young 
English Learners (ELs). We compare scores on a standardized English language 
proficiency assessment for 2,034 ELs from kindergarten through Grade 5 in the 
US who either used the CALL tool during the 2020-2021 academic school year 
(n=1,478) or did not (n=556). Descriptive analyses show larger scores for students 
who used the program than those who did not, and this was seen across student 
demographic subgroups. Statistical analyses reveal that this difference is significant 
even when accounting for student demographics and enrolled school and that 
greater program use is related to higher proficiency scores. The largest effects are 
seen for oral proficiency scores. The study raises implications for use of CALL tools 
in contexts where English is the language of education. 


Keywords: academic English, young language learners, oracy, oral proficiency. 


1. Introduction 


Within the US, children learning English as a second language readily acquire the 
social oral language used in informal contexts (e.g. with friends), but often do not 
acquire the language proficiency skills needed for success in content-area classes 
(Menken, Kleyn, & Chae, 2012). This ‘academic English’ is a formal register of 
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standard English and is integral for helping students interact in the classroom and 
successfully obtain and make use of subject matter information (Scarcella, 2003; 
Valdés, 2004). Furthermore, many ELs receive services that are not appropriate 
for their English proficiency level (Menken et al., 2012) and which often do not 
emphasize oral language skills (oracy, NASEM, 2017). 


CALL tools may help effectively address the academic language needs of ELs. 
To this extent, Lexia® English Language Development™ (Lexia English) was 
developed and released in 2020. Lexia English is a blended learning program for 
ELs in kindergarten through Grade 6 (K-6), and consists of an online computer 
program, teacher-led lessons, and student progress-monitoring data. The program 
consists primarily of speaking and listening activities, which utilize academic 
content (science, math, social studies, general knowledge) and are framed as 
simple, scripted conversations with program characters. An example is shown in 
Figure 1. 


Figure 1. Example program image 


| 


She walked between the teacher and the table. 


O 


The program is grounded in various theories of second language acquisition. 
Comprehensible input (Krashen, 1982) is presented via language frames, and an 
auto-placement tool places users at an appropriately challenging level. A speech 
recognition engine attuned to various English accents allows for users to practice 
speaking (Output Hypothesis; Swain, 1995), while corrective feedback is provided 
explicitly to learners (Interaction Hypothesis; Long, 1996). Furthermore, an 
adaptive program design provides students with explicit instruction and additional 
practice for activities they answer wrong twice in a row. 
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The aim of this study was to examine the impact of Lexia English on English 
language proficiency outcomes for students in Grades K-5. Our research 
questions were the following: Do students who use Lexia English score higher 
on standardized English language proficiency tests compared to their peers who 
do not use the program? How does the amount of program use impact English 
proficiency outcomes? 


2. Method 


One school district in the US signed a data exchange agreement with Lexia to 
share demographic and assessment data for 2,034 EL students in Grades K-S5 across 
21 schools. English proficiency scores were from the 2021 English Language 
Proficiency Assessment for California (ELPAC, scores on vertical scale from 
1,150-1,700 Grades K-2, 1,150-1,800 Grades 3-5) and included scores for the 
overall assessment and oral and written subdomains. 


All district EL students had access to Lexia English during the 2020-2021 school 
year, however there was considerable variation in the amount of program units 
completed (Range=1-1,163, M=85, SD=119) due to pandemic-induced changes 
to remote and hybrid schooling environments. We considered program users to be 
students who completed at least one unit (three to five minutes). 


We transformed student scores into grade-specific z-scores (interpreted as standard 
deviations, SDs) and statistically analyzed them using multiple linear regressions. 
We included 2021 ELPAC scores as the dependent variable, and dummy-coded 
predictor variables for Ll (Spanish, non-Spanish), gender (male, female), 
socioeconomic status (low SES, non-low SES), and enrolled school. 

3. Results and discussion 


Asummary of overall ELPAC scaled scores is shown in Table |. 


Table 1. Mean (SD) ELPAC scores by group 


Program Users Non-users 
All participants 1,487 (70) 1,472 (109) 
Gender 
Male (n=1,112) 1,486 (72) 1,463 (118) 
Female (n=922) 1,488 (68) 1,484 (94) 
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SES 

Low SES (n=937) 1,471 (63) 1,449 (100) 
Non-low SES (n=1,097) 1,502 (74) 1,487 (112) 
L1 Group* 

Spanish (n=896) 1,467 (68) 1,447 (96) 
Non-Spanish (n=1,138) 1,505 (68) 1,486 (113) 


*Students spoke 50 different L1s, but we grouped this as a two-level variable for statistical power. 


As seen in Table 1, ELPAC scores were higher for Lexia English users than non- 
Lexia English users, and this pattern is seen for all student demographic subgroups. 


Figure 2. Average ELPAC score by domain and standard error bars 


 Lexia English user 


non-Lexia English user 
1490 a a 


1480 


1470 


1460 1472 1488 1470 1487 1473 


Overall Oral Written 


As seen in Figure 2, Lexia English users scored higher than non-Lexia English 
users in both the oral and written domains. Regression analyses confirmed these 
patterns: Lexia English users scored significantly higher than non-Lexia English 
users on the overall (B=0.193, SE=.051, p<.001; R?=0.192, p<.001), oral (B=0.219, 
SE=.052, p<.001; R?=0.155, p<.001), and written ELPAC (B=0.149, SE=.051, 
p=.004; R?=0.191, p<.001). The largest effect was seen for the oral domain, in line 
with the program’s focus. 


We also examined just the Lexia English users and incorporated a predictor variable 
for the number of program units completed. This revealed that a greater number of 
program units completed was significantly related to higher scores on the overall 
ELPAC (B=0.045, SE=.015, p=.003; R?=0.112, p<.001), such that completion of 
22 units was associated with a one point scale score increase. 
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Overall, these results demonstrate how explicit language instruction and targeted 
oral language practice can be incorporated into CALL technology to support 
academic language and oracy development for young ELs. Many existing 
English language ed-tech tools are designed for the acquisition of reading and 
writing skills or for acquiring English in contexts where it is not the primary 
language of education (e.g. Bang, Olander, & Lenihan, 2020). Thus, this is a novel 
contribution toward applying CALL tools across different contexts. Additionally, 
the diverse sample may suggest that the positive findings could generalize the use 
of similar tools in contexts where English is the primary language of education 
and society. 


The results corroborate positive findings seen for personalized learning affordances 
of adaptive learning systems (Slavuj, MeStrovic, & Kovacic, 2017), and dedicated 
attention to speaking and listening activities in (system-guided) dialogue-based 
CALL systems (Bibauw, Francois, van den Noortgate, & Desmet, 2022). The 
program is not meant to replace real-life dialogue, but rather complements 
classroom instruction by providing ELs a low-anxiety environment to strengthen 
their English language skills via guided, self-paced interactions with program 
characters (see Bibauw et al., 2022). Finally, positive findings were seen for both 
written and oral domains, in line with research highlighting the importance of oral 
language skills in reading/writing (NASEM, 2017). 


4. Conclusions 


This study shows early evidence that a CALL tool focused on oral language and 
academic English for young ELs may lead to higher English language proficiency, 
particularly as students complete a greater number of program units. We cannot 
claim that the program causes this as we did not control for students’ prior year 
achievement scores (due to limited data availability). Additionally, it remains to 
be seen how educators’ use of teacher-led lessons and student progress-monitoring 
data impacts student outcomes. Overall, the study serves as a promising first step 
toward understanding the efficacy of a new CALL tool. 
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Abstract. This article provides evidence of critical factors of student retention in 
Language Massive Open Online Courses (LMOOCs). The study used multiple 
sources: tracked retention data (n=43,000), survey data in correlation with tracking 
data (n=400), and qualitative data (174 informants) from a survey (Fridriksdottir, 
2018, 2021a, 2021b). The data came from learners in seven courses and across 
different delivery modes in the LMOOC program Icelandic Online (IOL, https:// 
icelandiconline.com). First, analysis of the tracking data revealed low overall 
completion rates and that retention was highest in the blended mode of delivery. The 
analysis also uncovered a specific dropout pattern and pattern of user engagement. 
Second, the survey study identified six content-related and four tutor-related 
engagement factors and that leamers’ differing goals for course participation 
influenced retention. Finally, qualitative data analysis revealed intrinsic and extrinsic 
motivational factors responsible for course completion, whereas non-course-related 
factors affected retention among dropouts. Overall, the study identified multi-ranged 


determinants of student retention. 


Keywords: LMOOC retention, content factors, tutored factors, Icelandic Online. 


1. Introduction 


This paper briefly summarizes a research project on LMOOCs presented in 
Fridriksdottir’s (2018, 2021a, 2021b) previous articles. In this study, critical 
factors of retention in LMOOCs were considered within the context of IOL, a 
free and open web-based online program for L2 Icelandic learners developed at 
the University of Iceland. This Computer Assisted Language Learning (CALL) 
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program offers seven free, self-directed, interactive courses (CEFR A1l-C1) and 
features an integrated tracking system that monitors students’ behavior from the 
time they enter a course until they leave. Around 270,000 learners worldwide have 
attended the program since the launch of the first course in 2004. IOL’s design relies 
on diverse second-language acquisition pedagogical principles (Arnbjérnsdottir, 
2004; Arnbjérnsdottir, Fridriksdottir, & Bédi, 2020) which are evident in the 
content structure and the strategies used for instruction and practice. The courses 
are delivered in three learning modes; an open self-directed mode, a blended mode, 
and a distance mode. A human tutor offers guidance in the blended and distance 
modes, while the open self-directed modes do not. 


This study addresses the issues of low retention rates (Jordan, 2015) and attrition 
patterns (Reich, 2014) in LMOOCs in relation to a course’s learning materials 
and the lack of empirical studies based on large-scale, long-term, computer-based 
tracking data (Gillespie, 2020). The study also addresses concerns that LMOOCs 
lack engaging pedagogy and design strategies for the L2 learners (Colpaert, 2014; 
Sokolik, 2014). They also lack direct guidance and support from a tutor in such 
a learning environment (Ross et al., 2014). These issues might explain the high 
attrition rates in LMOOCs. More evidence is needed from LMOOC learners 
themselves, who may provide significant insight into their own engagement 
patterns (Doiz, Lasagabaster, & Sierra, 2014). 


2. Methods 


This research study employed mixed methods. First, tracked retention data 
(n=43,468) was collected over eight years from learners in all seven IOL courses 
and the three delivery modes of two courses to gain an insight into student retention 
and overall online behavior in IOL. Data mining and Learning Analytics (LA) were 
employed in the analysis of this data. 


Second, a survey instrument and a questionnaire were used to provide information 
about learners’ experiences of various course-specific elements and other 
motivational factors. Six content-related factors were investigated: curated and 
sequenced course structure, clear and salient learning objectives, gradual and 
scaffolded presentation of input, continuing storylines, form-focused and scaffolded 
presentation of grammar, and variety in types of learning objects. Additionally, 
four tutor-specific factors were examined: a detailed introduction to the program, 
private interaction with the tutor, a set syllabus, and overall tutor support during 
the course (Fridriksdottir, 2021c, p. 27). The factor of learners’ initial goals for 
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participation was also explored. Survey data (n=400) and retention data (n=400) 
on each participant were analyzed separately and then merged. 


Finally, an open-ended questionnaire was distributed through the survey among 
learners with an aim toward generating insight into why learners either complete a 
course or disengage earlier. This data (174 informants) was analyzed for common 
themes. 


3. Results 


Firstly, the tracking data analysis (n=43,468) showed low completion rates across 
all courses and delivery modes (2.4 to 18.2%), and that the blended learning modes 
were more effective in retaining learners than the other delivery modes studied. 
Furthermore, through the mining of this data and the use of LA, the study identified 
a pattern of attrition among non-completers as well as a pattern of user engagement 
across all courses and modes. The findings showed that students frequently drop 
out early in these courses, but that they also may disengage toward the end 
(Fridriksdottir, 2018, 202 1c). 


Secondly, the survey study (n=400) identified six content-related factors that most 
participants considered important for their engagement with the course, i.e. curated 
and sequenced course structure, clear and salient learning objectives, gradual and 
scaffolded presentation of input, variety in types of learning objects, continuing 
storylines, and form-focused and scaffolded presentation of grammar. When the 
survey data was measured against the tracking data, three of the content factors 
were found to have a positive impact on student retention: curated and sequenced 
course structure, gradual and scaffolded presentation of input, and variety in types 
of learning objects. Similarly, most participants considered the four tutor-related 
factors important for their engagement, i.e. set syllabus, private interaction with the 
tutor, detailed introduction of the program, and overall tutor support. All of these 
factors had a positive impact on retention in the blended mode, but none of them 
did in the distance mode (Fridriksdottir, 202 1a, 2021b, 2021c). 


Thirdly, the results showed that over half of the learners (n=400) began the program 
with the intention of completing the full course while the rest did not, and that the 
goal to complete a course had a significant impact on actual completion. 


Finally, analysis of the qualitative data (112 informants) from the course completers 
revealed various motivating factors for continuing with the course, such as 
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interesting course material or an interest in the language or culture. Furthermore, 
statements from learners (62 informants) who intended to complete a course 
but disengaged earlier show that factors unrelated to the course, such as time 
constraints, affected retention (Fridriksdottir, 2021a, 2021b, 2021c). 


4. Discussion and conclusions 


Collectively, this study contributes wide-ranging data from different sources on the 
engagement behavior and perspectives of LMOOC learners in the context of their 
learning materials. This large-scale, long-term, empirical tracking study provides 
detailed information on learners’ online behavior, the timing of student attrition, 
as well as the extent to which non-completers engage with the course material. 
This finding highlights the benefits of exploring tracked progress within LMOOCs. 
Furthermore, the survey study’s findings that IOL’s engagement strategies and the 
presence of a tutor may be crucial for learner engagement highlight the value of 
CALL design and tutor support in LMOOCs. Results also showed that learners join 
a course with different goals for participation, which bears discussion in relation to 
retention. The qualitative data analysis, moreover, sheds light on a diverse group of 
students in this learning environment and revealed broad individual perspectives 
on critical factors of LMOOC retention and attrition. This study informs design 
directions for course developers and instructors in LMOOCs. Future research 
possibilities based on the findings of this study include an investigation of the 
benefit of supplementary resources in LMOOCs for student retention and learning 
outcomes. 
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Abstract. The use of chatbots in language learning has been on the rise. In recent 
Computer-Assisted Language Learning (CALL) research, there is a consensus that 
rule-based, scripted voice systems are optimal for language learning. Such systems 
integrate well into instructed language learning in that interactions with the user 
are predictable and controlled. Open, Al-based voice systems (such as in personal 
assistants like Siri) do not provide that degree of task-oriented learning. However, 
the argument is made here that they have the potential to provide open-ended 
conversational practice and language development which aligns with an ecological, 
usage-based perspective on language development. 


Keywords: chatbots, artificial intelligence, voice systems, intelligent personal 


assistants. 


1. Introduction 


Most dialogue-based projects in CALL involving voice have utilized rule-based, 
limited-domain systems (Fryer, Coniam, Carpenter, & Lapusneanu, 2020). Such 
systems have been available in text-based dialogue systems going back to the 
1960’s. Experimentation with these systems for language learning began in the 
1980’s, particularly through work in tutorial CALL. More recently, a number of 
chatbots for language learning have been released, including Mondly and Eggbun 
(Alm & Nkomo, 2020). General-purpose chatbots have also been used, such as 
Cleverbot (Fryer et al., 2020). Interactions in such systems are based on scripts 
deployed as decision trees, which supply predetermined responses to user input. 
This provides a high level of control, leading to relatively predictable exchanges. 
Using such voice systems, particular areas of language study can be targeted, and 
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the level of language adjusted for learners with different degrees of proficiency. 
This kind of controlled practice aligns with skill acquisition theory and has proven 
to be a widely used approach in integrating chatbots into structured language 
learning environments. Free-flowing conversations with a bot can be a frustrating 
experience, as many studies have shown. Bibauw, Francois, and Desmet (2019) 
argue that “free dialogue should not be seen as the ultimate target of dialogue- 
based CALL” (p. 856). That position is echoed in most studies of voice systems in 
CALL. This paper argues, however, that open-ended conversations with a spoken 
dialogue system should not be quickly dismissed as a viable alternative to scripted 
systems. In particular, Intelligent Personal Assistants (IPA’s), such as Alexa, offer 
the potential to combine the advantages of language practice through open-ended 
conversation with features enhancing both learner willingness to communicate and 
implicit, usage-based learning. 


2. Al-based systems 


Dialogue management in scripted systems is handled in a variety of ways, through 
implicit or explicit mechanisms. Bibauw et al. (2019) offer a typology of constraints 
used in dialogue-based CALL. Typically, exchanges are based on a question- 
answer format, with responses controlled with multiple choice or other closed 
response options. In that way, the systems supply pattern drills in chosen language 
use situations, with limited opportunities for multiturn exchanges. They are not 
designed to provide practice in meaningful exchanges outside the subject area and 
language limits imposed by the system. Such systems function as tutors, carrying 
out a specific pedagogical role. While this is certainly useful for language learners, 
particularly at early stages of development, chatbots offering open conversation 
can do more, namely provide both practice in linguistic forms and socialization 
into situated language use. That can be especially useful at later stages of language 
development. Recent developments in voice technologies, fueled by advances in AI 
and big data collection have led to dramatically improved performance of general- 
purpose chatbots, such as Google Assistant. These IPAs are built into smartphones 
and enable users to access through voice many services available online including 
search or translation. They are increasingly becoming available in other connected 
devices such as earbuds, home speakers, and car consoles. This developing Internet 
of Things (IoT) heralds the arrival of ambient intelligence, with voice services 
available almost anywhere, at least for affluent populations. This represents one 
of the singular advantages of IPAs for language learning, namely that rather than 
being a scheduled learning task, they integrate into the everyday lives of learners 
as available second language partners. 
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While voice services are available in the most widely spoken languages, not every 
language is represented. The range of services may be limited by language as 
well, with the most up-to-date and robust support for speakers of English. Voice 
recognition services are designed for mainstream speakers of the language and will 
likely struggle with non-standard speech including dialects and learner language. 
Nevertheless, there has been considerable experimentation with the use of IPAs 
in language learning, with generally positive results (Bibauw, Francois, Van 
den Noortgate, & Desmet, 2022). Their use has been shown to be effective for 
novice learners, who are able to practice pronunciation and basic conversations. 
The systems can serve as models of expert speakers, and, compared to humans, 
possess infinite patience, allowing for extensive trial and error without judgment. 
In that way, they have been shown to help overcome anxiety and encourage a 
willingness to communicate (Alm & Nkomo, 2020). Most studies report on IPA 
use in instructed learning environments; little research has been published on 
their use by autonomous learners or in informal learning settings. Yet that use is 
likely to hold considerable promise for both independent language learning and 
language maintenance. That will particularly be the case as more personal devices 
such as smartglasses are released, which allow interactive engagement with users’ 
surroundings mediated by language. 


3. Expanding voice systems for language learning 


While IPAs in their current configurations have been shown to be potentially 
beneficial to language learners, there are developments and add-ons that can make 
them more useful for that purpose. One of the options is to add functionality to a 
commercial system through a third-party app. That may be possible if an appropriate 
API service is available or if the system is set up to integrate add-ons, as is the case 
with Alexa. That Amazon-created IPA allows for ‘skills’ to be added to its core 
functions. Several language learning apps have been created in that way (Maria, 
2021). One of the more promising directions for chatbots is to use and customize 
a social bot. A prime example is XiaoIce from Microsoft Asia. While mainstream 
IPAs are designed to function as transactional agents, i.e. to reply succinctly to user 
queries, social bots are programmed to serve as artificial companions. They are 
endowed with a distinct personality profile and are capable of engaging in multiturn 
conversations. One of the enablers of that functionality is the maintenance in the AI 
system of a distinct user profile. This allows a chatbot to recall and use information 
from prior dialogues. Xiaolce is typical of social bots in that it is not just reactive, 
but proactive, asking follow-up questions, and even suggesting conversation topics. 
Studies using Xiaolce describe conversations lasting over half an hour (Zhou, Gao, 
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Li, & Shum, 2020). In contrast, IPAs currently maintain a minimal user profile, 
usually limited to information about subscribed services. On the other hand, they 
are moving in the direction of developing greater social skills, with some ability to 
engage in small talk, improved dialogue management, and synthetic voices which 
sound much less robotic. 


In order for chatbots to move from the role of tutor to that of conversation partner, 
the AI system needs to be able to converse on a broad array of topics. One of the 
developments in AI that is moving in that direction are large language models such 
as LaMDA (from Google) and GPT-3 (from OpenAI). These are systems based 
on incredibly large datasets, which are converted into artificial neural nets using 
machine learning (Godwin-Jones, 2021). They have been shown to be capable of 
generating texts on a large variety of topics, in different genres, and in various 
languages. Hybrid systems, built upon extending existing large language models, 
have the potential to combine the advantages of language practice through open- 
ended conversation with features enhancing both learner motivation and implicit, 
usage-based learning. 


4. Conclusion 


Using a mainstream IPA, deployed through a dedicated app or a commonly used 
messaging system, shifts the dynamics of exchanges from an artificial academic 
setting to an experience integrated into everyday life. That is likely to make the service 
more universally available (including through IoT devices) and enhance motivation. 
That persistent and individualized, always-available status holds the promise of new 
options for language learning/maintenance, especially for autonomous learning. This 
aligns with ecological approaches to CALL as well as emergentist views of second 
language acquisition, which move away from linear and pre-defined pathways to 
language learning in favor of a more dynamic and holistic approach. 


References 


Alm, A., & Nkomo, L. (2020). Chatbot experiences of informal language learners: a sentiment 
analysis. International Journal of Computer-Assisted Language Learning and Teaching 
(IJCALLT), 10(4), 51-65. https://ideas.repec.org/a/igg/jcallt/v 10y202014p5 1-65. html 

Bibauw, S., Francois, T., & Desmet, P. (2019). Discussing with a computer to practice a foreign 
language: research synthesis and conceptual framework of dialogue-based CALL. Computer 
Assisted Language Learning, 32(8), 827-877. https://doi.org/10.1080/09588221.2018.1535508 


127 


Robert Godwin-Jones 


Bibauw, S., Francois, T., Van den Noortgate, W., & Desmet, P. (2022). Dialogue systems for 
language learning: a meta-analysis. Language Learning & Technology, 26(1), 1-24. https:// 
www.lltjournal.org/item/10125-73488/ 

Fryer, L., Coniam, D., Carpenter, R., & Lapusneanu, D. (2020). Bots for language learning now: 
current and future directions. Language Learning & Technology, 24(2), 8-22. https://www. 
Iltjournal.org/item/10125-44719/ 

Godwin-Jones, R. (2021). Big data and language learning: opportunities and challenges. 
Language Learning & Technology, 25(1), 4-19. https:/Avww.lltjournal.org/item/10125- 
44747/ 

Maria, A. (2021). Got an Alexa? You've got a polyglot tutor that can teach you a language. 
FluentU. https://www.fluentu.com/blog/can-alexa-teach-languages/ 

Zhou, L., Gao, J., Li, D., & Shum, H.-Y. (2020.) The design and implementation of Xiaolce, 
an empathetic social chatbot. Computational Linguistics, 46(1), 1-35. https://arxiv.org/ 
abs/1812.08989 


128 


EU 


Verbal and nonverbal communication 
in high-immersion virtual reality 
for language learners 


Alice Gruber! and Regina Kaplan-Rakowski* 


Abstract. Virtual Reality (VR) offers language learners a valuable environment for 
practicing language skills and other aspects essential for language development, 
interaction, and negotiation of meaning. For example, speaking practice in VR 
using avatars can reduce speaking anxiety and increase users’ sense of agency. 
Social spaces in VR present possibilities for verbal and nonverbal communication. 
This conceptual paper discusses the advantages and drawbacks of both types of 
communication in high-immersion VR and the implications for language learners. 
Language learners rely on multiple modes of communication, including nonverbal 
behavior such as gestures. However, VR often lacks behavioral anthropomorphism, 
i.e. when an avatar cannot speak, move, or act in a human-like way, which may 


negatively impact communication, especially at beginner level. 


Keywords: high-immersion VR, language learning, verbal and nonverbal 


communication, online social spaces. 


1. Verbal and nonverbal communication 


While interacting, interlocutors depend on Verbal Communication (VC) and 
NonVerbal Communication (NVC). VC can be spoken or written and uses words 
(i.e. language, see Rocci & de Saussure, 2016). Speech in a face-to-face format 
differs from that in VR format because faces are shielded through an avatar in 
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VR (i.e. a representation of self in virtual settings). Written communication in VR 
spaces typically takes place using chats or personal messages. 


NVC, such as facial behavior, gaze, gestural behavior, and spatial behavior, 
plays an important role in human interaction (Chun, Karimi, & Safosa, 2022). 
Facial expressions and gaze reveal information, emotional states, attention, 
and intention (Maloney, Freeman, & Wohn, 2020). Gestural behavior including 
hand movements can support and replace verbal behavior. Spatial behavior, 
or proxemics, refers to the measurable distances between interlocutors and 
is culturally dependent (Maloney et al., 2020). NVC can convey information 
regarding a person’s willingness to communicate and to listen (Barkai, 1990). 
It can also regulate interactions, for instance, when nodding is used to signal 
turn-taking (Aburumman, Gillies, Ward, & Hamilton, 2022). In a language 
learning context, NVC is particularly important because audiovisual presentation 
facilitates language learners’ perception and recognition of foreign sounds (Davis 
& Kim, 2001). In this paper, we use the terms nonverbal communication and 
nonverbal behavior interchangeably. 


2. VR-assisted language learning and verbal 
and nonverbal communication 


Given the growing interest in VR-assisted language learning (Dhimolea, Kaplan- 
Rakowski, & Lin, 2022) and the fact that communication is key in language 
learning, our article discusses how VC and NVC take place within various VR 
spaces and what possible shortcomings of VR technology currently exist regarding 
communication. 


We focus on high-immersion VR, which is defined as “computer-generated 360° 
virtual space that can be perceived as being spatially realistic, due to the high 
immersion afforded by a head-mounted device” (Kaplan-Rakowski & Gruber, 
2019, p. 552). Wearing a head-mounted device, known as a VR headset, offers 
embodiment, sense of presence, and immersion. High-immersion VR allows 
for an embodied interaction and a wide range of NVC (Maloney et al., 2020). 
Interlocutors in VR can use a haptic system (i.e. virtual hands), which allows 
for making gestures and acting in other ways that feel natural and, consequently, 
increase the embodied cognition. Social VR applications (e.g. vTime and 
BigScreen VR) enable avatar-mediated interaction between users and often 
include novel options for nonverbal behavior using virtual emojis (see Figure | 
below) to compensate for the lack of facial expressions. 
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VR-specific interactional patterns and norms may develop and influence each 
other (Ahlers, Lazovic, Schweiger, & Senkbeil, 2020). High-immersion social VR 
can impact interaction regarding topic choice (Gruber, Canto, & Jauregi-Ondarra, 
forthcoming) and, as a result, can influence learners’ lexical choice. While VC 
depends on users’ own voice and their written input, NVC is conveyed through 
users’ avatars with limited gestures and animations that are often pre-programmed 
(Maloney et al., 2020). 


In language learning contexts, social VR apps can be used for a variety of 
communication constellations such as one-on-one tutoring (Kaplan-Rakowski & 
Gruber, 2021), larger group tutoring (as in the case of Immerse), collaborative 
international learning, and intercultural encounters (Ahlers et al., 2020; Gruber et 
al., forthcoming; Jauregi-Ondarra, Gruber, & Canto, 2021). In these interactions, 
users’ voices are their own, and users are personified as customized avatars 
(see Figure 2 below). While users of social VR apps in noneducational settings 
sometimes choose to rely only on NVC for privacy and disable the voice function 
(Maloney et al., 2020), language learners using social VR apps are typically 
expected to complete tasks orally in the target language. 


Before 2022, VR headsets could only convey subtle emotional cues until the 
release of Meta Quest Pro, which introduced full face and eye tracking allowing to 
facilitate NVC. Although NVC is integral to language learning, research on verbal 
behavior and NVC is limited. 


Figure 1. Virtual emojis (https://hubs.mozilla.com/docs/hubs-features. html) 
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Figure 2. Collaborative interaction in VR 


Although NVC is integral to language learning, research on verbal behavior 
and NVC is limited. Peixoto, Melo, Cabral, and Bessa (2021) explored the 
avatars’ body animations and lip synchronization on listening comprehension 
in an English as a foreign language setting. The results indicate that avatars 
with realistic animations, movements, and lip synchronization positively affect 
the users’ sense of presence, learning, and overall experience. A lack of lip 
synchronization during avatar-to-avatar communication can negatively affect 
language comprehension (Kaplan-Rakowski & Gruber, 2021) because language 
learners must rely mostly on live voice input and paralinguistic features such as 
nonverbal vocal qualities. 


3. Implications, future directions, and conclusion 


Educators need to consider learners’ limited choices as well as the novel options 
available regarding verbal and nonverbal behavior in social VR applications and 
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their impact on learners’ interactions. Commercial VR apps do not typically offer 
face tracking, which can make communication more challenging. The use of VR- 
inherent NVC options such as virtual emojis may influence and change learners’ 
interactional patterns, including their verbal behavior, and their use of strategies 
to communicate successfully. The lack of lip-reading opportunities may mean 
that learners rely on and, thus, train their listening skills to decipher spoken text. 
Although research indicates that Foreign Language Anxiety (FLA) can be reduced 
in VR (Gruber & Kaplan-Rakowski, 2020, 2022), the need to predominantly attend 
to verbal behavior and the lack of lip synchronization may impact FLA. Research 
is required to understand the effect of VR-inherent verbal and nonverbal behavior 
options on FLA. 


Furthermore, no study has examined whether and how language learners 
compensate for the lack of facial expressions, especially at lower proficiency 
levels. Little attention has been paid to language learners’ negotiation of meaning 
and compensation strategies when faced with time lags and technological glitches. 
The possible effect of limited nonverbal cues on language learners’ cognitive load 
and social presence during VR interactions should be explored. Furthermore, 
research is needed to understand whether and how spatial behavior in VR and the 
use of virtual emojis are culture-dependent and whether language learners notice 
cultural differences in such context. 
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Abstract. In this paper, we report on a study that took place during the COVID-19 
pandemic and for which we interviewed ten experienced, university level, language 
instructors about their digital practices as they found themselves teaching in an 
Emergency Remote Teaching (ERT) mode. The study sought to describe how, 
through their professional activities and experiences, they developed new and/ 
or further online competencies and how the ERT context brought them to rethink 
their pedagogical practices and namely, their Written Corrective Feedback (WCF). 
Our results show that language instructors’ digital competencies are on a dynamic 
continuum of changes with some who faced challenges, while others sought 
opportunities or provided solutions during that unprecedented period. An adapted 
version of the SAMR (Substitution, Augmentation, Modification, and Redefinition) 
model (Puentedura, 2010) is suggested, which takes into consideration this ERT 
context. 


Keywords: emergency remote teaching, professional didactics, digital competencies, 
language instructors, written corrective feedback. 


1. Introduction 


The COVID-19 pandemic precipitated language instructors into an ERT mode (Jin, 
Deifell, & Angus, 2021), finding themselves teaching in an unfamiliar professional 
context for which they were not prepared. Consequently, they had no choice but 
to cope, develop some online competencies (Cf. Hampel & Stickler, 2015), and 
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adapt their pedagogy to be able to continue to meet their expected and institutional 
professional standards, as well as their language courses’ requirements and 
students’ needs. 


Adopting a ‘professional didactics’ stance (Pastré, Mayen, & Vergnaud, 2006), 
we designed our study seeking to describe how, through their professional 
activities and experiences, language instructors developed new and/or further 
online competencies; how the ERT context brought them to rethink their 
pedagogical practices and namely, their WCF ones (Cf. Hamel & Bibeau, 
2021). Professional didactics view and define professional work as training and 
transformation opportunities (Mayen, 2012, p. 60), both within and outside of 
formal professional development training contexts. It is an intersection of work, 
learning, and training. It aligns well with the ERT context where training was 
done rapidly, in vivo. 


This study is part of a larger CALL design and development project for which 
we are currently prototyping a digital tool to support and optimize language 
instructors’ digital WCF practices (Hamel & Bibeau, 2021; Hamel, Slavkov, 
Inkpen, & Xiao, 2016). We investigated these practices in order to better 
understand the ergonomics of the ‘teacher-task-tool’ interaction in that specific 
digital context (Caws & Hamel, 2016) and to identify desirable affordances 
(Gibson, 1979) for our prototype. 


2. Method 


In October 2021, we conducted ten semi-guided interviews with experienced 
language instructors from five universities in Canada and one in Brazil. These 
interviews followed two questionnaires that were distributed before (2019) and 
during the pandemic (February 2020), which focused on language instructors’ 
digital practices in general, and WCF practices specifically (Hamel & Bibeau, 
2021). The interviews were in two parts: first, the language instructors describing 
their ERT context, coping pedagogy, and developed competencies, and second, 
focusing on their specific WCF practices: namely, digital written tasks, tools, 
and feedback strategies as well as how they envision their post-pandemic WCF 
practices. 


The interview data were analyzed thematically with NVivo, with reference to the 
SAMR framework (Puentedura, 2010), which allowed for coding, word clouds, 
and a careful reading for the patterns to appear. 
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3. Results and discussion 


Ata glance, our results show that our ten language instructors’ digital competencies 
can be situated on a dynamic continuum of changes (Cf. Figure 1). The ERT 
context has brought challenges for ‘old schoolers’ (Shannon and Marielle) who, 
with reference to the SAMR model, show substitution and adaptation digital 
competencies; opportunities for those ‘on the move’ (Dina, Marie-Anne, and 
Marie) showing modification and redefinition digital competencies; and/or 
solutions for the ‘tech inclined’ (Eloise, Beatrice, and Rosalie) who have gone 
beyond redefinition. 


Figure 1. Language instructors’ digital competencies on a dynamic continuum 


of changes 
SOLUTIONS OPPORTUNITIES CHALLENGES 
al Redefinition Modification Adaptation Substitution 
- eo 
tech-inclined on the move old school 
Elere Marie-Anne Mariell 
Beatrice Dina “Laure, Elizon ie 
. Marie Shannon 
Rosalie 


Challenges identified by the ‘old schoolers’ (n=2) throughout the pandemic had 
to do, for instance, with not finding the comfort of their traditional practices. 
One instructor said that she found the time required to learn a new digital tool 
daunting; she also feared ‘wasting time’ in the classroom if things did not go 
smoothly. Other challenges identified were digital fatigue, lack of opportunities 
for synchronous interactions with students, and lack of engagement from the 
students, especially concerning the feedback they received. This category of 
instructors took advantage of informal opportunities to develop their digital 
practice, for example by reaching out to colleagues and others in their personal 
network, as well as training opportunities provided by their institution. They 
did not identify an evolution or change to their practices over the course of 
the pandemic. One instructor, after this year and a half of online teaching, still 
described herself as ‘very hesitant’ (Shannon). 


Most of the instructors (n=5) could be placed in the middle category, representing 
those ‘on the move’, that is, willing to change. While several instructors expressed 
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that they found the initial switch to ERT ‘a challenge’, they also felt that they 
‘managed’. The situation provided the opportunity to empathize with their 
students, for example: “ca m'a amené beaucoup 4 me mettre dans la perspective 
des apprenants. Davantage, je dirais” [it has helped me a lot in putting myself in 
the perspective of the learners. More I would say] (Dina). These instructors were 
able to gain confidence in the new modality, discover new digital tools, and provide 
more detailed, interactive, and multimodal feedback. They reported using a mix of 
methods for their own professional development. This included expanding their 
use of existing competencies, as well as trial and error, which resulted in their being 
about to challenge the status quo in their university language teaching practices. 
Overall, the instructors who were placed in this category reported a positive self- 
perception when teaching online. 


In the category of the tech inclined (n=3), the instructors already had a high level 
of comfort with digital tools and tasks before ERT and found themselves “not in 
panic mode” (Rosalie). Instead, their familiarity with digital tools and tasks meant 
they could select from those that they were already familiar with. For example, 
Rosalie said that she “made a deliberate choice to use less technology because 
of the students”. The instructors in this category still engaged in professional 
development, but since they were already starting from a place of comfort with 
technology, they identified less evolution: “I don’t see myself changed” (Rosalie). 
Another instructor perceived herself as “comfortable, confident, and efficient” 
(Beatrice) when teaching online. 


These three categories that we identified align with Jiang and Yu’s (2021) profiles 
of change in teachers’ feedback practices (positive, negative, and unchanged) while 
bringing continuity in the equation. The teachers are, after all, dynamic agents in 
their own practice. 


Teaching in the ERT context has been transformative for some language instructors. 
This was especially noticeable from those who we placed in the middle as on-the- 
move. They had a sufficient level of ease with digital tools and tasks so that they 
were able to significantly modify, that is, ‘redefine’ their teaching practice. At the 
same time, the results show that the SAMR model does not completely capture the 
experience of instructors in ERT, particularly those who are already tech inclined. 
Hence, we suggest adding an extra step to the SAMR model called Refining, with 
a sidebar description of Affirmation (Figure 2). This step describes the reality of 
instructors who already exploited technology for various affordances, had it fully 
integrated into their pedagogical practice, and had enough tools available to choose 
those most efficient and best adapted to their context and their students’ needs. 
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Figure 2. An adapted SAMR-+R model considering the ERT context 


AFFIRMATION 


MODIFICATION 
Technology allows for significant task 
redesign. 


TRANSFORMATION 


AUGMENTATION 


Technology acts as a direct substitute, with 
functional improvement. 


SUBSTITUTION 


ENHANCEMENT 


Technology acts as a direct substitute, with 
no functional improvement. 


4. Conclusion 


Overall, the results reveal language instructors’ perceptions of themselves as 
teaching professionals in their ERT context: their strengths and weaknesses, 
challenges and needs. It also shows how some redesigned their learning tasks, 
making most of the dynamic and collaborative affordances of digital tools and how 
they engaged in more bidirectional WCF practices with their learners, enhanced by 
multimodality. It will be interesting to conduct a further study on these language 
instructors’ post-covid digital practices and pedagogy. 
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Abstract. The attitudes of student-teachers to Computer Assisted Language Learning 
(CALL) remains an under-explored topic in research literature. This mixed-methods 
study engaged student-teachers in focus group discussions at commencement and 
completion of Initial Teacher Education (ITE). These student-teachers were situated 
across England, Scotland, Wales, and Northern Ireland. Student-teachers’ beliefs 
regarding the perceived value of CALL experienced a considerable transformation 
between initial and follow-up focus group interactions. Beginning student-teachers 
collectively portrayed CALL in a complimentary light, with 67.5% of participant 
dialogue positively coded via thematic analysis. This contrasts with follow-up 
student-teacher discourse, where only 38.55% of nodes in NVivo were positively 
coded. Overall, belief changes were most marked for positive categories of CALL. 
Thus, maintaining optimistic opinions to CALL could be a worthwhile endeavour 
to promote student-teacher acceptance of digital technologies in language learning. 


Keywords: CALL, student-teachers, attitudes, perceived usefulness. 


4. Introduction 


The primary aim of this research was to investigate student-teachers’ attitudes to 
CALL. Researchers have stipulated the need for supplementary studies on this 
population’s multidimensional relationship with CALL (Brun & Hinostroza, 2014, 
p. 236). This study details the cognitions of student-teachers to CALL over the 
duration of teacher education. This was from commencement to completion of 
their one-year Modern Language (ML) ITE programme. This research spanned 
across the four nations of the UK (England, Scotland, Wales, and Northern Ireland). 
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2. Method 


The geographical distribution of ITE establishments was an instrumental 
consideration in this study. It was vital to represent the UK and its regional diversity. 
Universities were initially targeted by disseminating an electronic questionnaire 
to teacher educators at every ML ITE institution in the UK. Responding teacher 
educators opted in/out of the research project. A focus group methodology was 
engaged in this investigation to understand student-teachers’ views on CALL 
from September 2020 to June 2021. These focus groups were conducted with 60 
student-teachers enrolled in seven ITE programmes. All focus groups occurred in 
the virtual environment due to COVID-19 restrictions. Thematic analysis ensued 
in the aftermath of focus group transcriptions. This analysis focused on meaning, 
patterns, and themes within the research data (Nastasi & Borja, 2017, p. 64). The 
six stages of Braun and Clarke’s (2006) thematic analysis enriched knowledge and 
understanding of CALL phenomena for the target population (Guest, Macqueen, 
& Namey, 2012, p. 161). 


3. Results and discussion 


Thematic analysis exhibited many forms of CALL usefulness for student-teachers 
enrolled on Postgraduate Certificate or Diploma in Education (PGCE/PGDE) 
courses. A total of 314 (initial) and 345 (follow-up) views of CALL usefulness were 
individually captured in the NVivo software. There were 345 positive (52.35%), 
84 mixed (12.75%), and 230 negative (34.90%) direct quotations and interjections 
of CALL worth from participants across both focus group stages. Student-teachers’ 
words, phrases, and language were conceptually categorised into 37 corresponding 
parent nodes of CALL value. The prevalence of these themes of CALL usefulness 
changed between focus group sessions. This is briefly summarised in Table |. 
Table 1. The total number of positive, negative, and mixed themes to CALL 
usefulness 


Number of Parent Nodes/Themes Corresponding to CALL Usefulness 


Positive 


Mixed 


Negative 


Total 


Initial focus groups 


19 


5 


13 


37 


Follow-up focus groups 


12 


4 


10 


26 


This thematic coding of CALL usefulness in NVivo is presented simply in Figure |. 
This qualitative data analysis methodology facilitated a strategic and systematic 


approach to understand the attitudes of student-teachers to CALL. 
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Figure 1. The process of qualitative thematic analysis in NVivo 


Initial focus groups (37 themes/ parent 
nodes; 314 child nodes/ coded text 
fragments) 


Follow-up focus groups (26 themes/ 
parent nodes; 345 child nodes/ coded 
text fragments) 


46.15% We 
were 
positive 


51.35% 
were 
positive 
parent 


67.52% were positive child nodes 
10.83% were mixed child nodes 


21.66% were negative child nodes 


38.55% were positive child nodes 
14.49% were mixed child nodes 
46.96% were negative child nodes 


Empirical data emerged from student-teacher dialogue by converting positive, 
negative, and mixed thematic codes of CALL into percentage occurrences within 
focus groups. The transformation of qualitative transcripts into quantitative data 
allowed the researcher to compare student-teachers’ responses across distinct 
regions meaningfully. In addition, the researcher observed belief variations to 
CALL usefulness between teacher education start and end. There was a statistically 
significant reduction of 28.97% in positive viewpoints of CALL usefulness from 
67.52% to 38.55%. Negative perceptions of CALL also increased by a quarter 
(25.30%) in final discussions from 21.66% to 46.96%. This compares to mixed 
perceptions to CALL value which remained remarkably consistent with only 
a 3.66% increase between focus group interactions. This displays that student- 
teachers’ positive and negative perceptions of CALL do not remain uniform. 
Instead, they are subject to considerable fluctuation throughout ITE. 


Student-teachers’ perceptions regarding the perceived value of CALL 
experienced a considerable transformation between initial and follow-up focus 
group interactions. Thematic analysis of focus group data identified 19 positive 
themes in initial discussions and 12 in follow-up conversations. This decline in 
positivity to CALL has been supported in literature as constructive perceptions 
are more susceptible to change (Zhelezovskaia, 2016, p. 48). This reinforces the 
role of systematic technology preparation for student-teachers to sustain positive 
perceptions to CALL. Negative opinions of CALL were higher at ITE finish 
as a point of comparison to complementary views. This has been evidenced in 
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similar investigations, which correlate increased teaching experience with negative 
perceptions about CALL (Gjelaj, Buza, Shatri, & Zabeli, 2020, p. 179). This 
finding has practical ramifications, as higher negativity correlates to lower CALL 
usage. Such attitudinal shifts highlight that this population’s perceptions about 
CALL are highly changeable, dynamic, and unfixed. Hence, student-teachers are 
likelier to display favourable yet fragile perceptions of CALL (Breiteig, Grevholm, 
& Kislenko, 2005, p. 4) Apart from these polarised categories of thinking, there 
were also uncertain reactions to CALL across opening and closing focus group 
discussions. However, thematic analysis showed that ambivalence to CALL was 
more consistent across the lifespan of teacher education. This is in direct contrast 
to approving and disapproving perceptions of CALL. 


Comprehending the favourable, ambivalent, and unfavourable perceptions of this 
population is of critical importance in making sense of why student-teachers 
behave the way that they do with digital technologies. This is because their 
attitude to CALL depicts a student-teacher’s state of readiness towards digital 
technology adoption. In fact, the belief systems of student-teachers are not 
isolated from their CALL classroom activities (Hell, Godhe, Wennas, & Brante, 
2021, p. 9). This means that student-teachers’ perceptions can only be understood 
from both their words and their actions. Thus, belief investigations within the 
field of teacher education are crucial as student-teachers’ ways of thinking about 
CALL correlate with classroom usage (Vidal-Hall, Flewitt, & Wyse, 2020, 
p. 168). These perceptions of CALL can determine student-teacher success or 
failure with digital technologies. 


4. Conclusions 


In conclusion, participants demonstrated complimentary cognitions of CALL at 
PGCE/PGDE onset. These favourable mindsets to CALL have been demonstrated 
across similar research investigations (Lowther, Inan, Strahl, & Ross, 2012, 
p. 23). This is a meaningful finding as positive beliefs have been associated with 
a greater likelihood of CALL classroom behaviours (Wilson & Donkin, 2021, 
p. 51). At the same time, however, approving beliefs do not always equate to 
actual CALL usage for classroom practitioners. However, this was not the case at 
ITE conclusion. Crucially, favourable opinions of CALL decreased in follow-up 
discussions. Literature has confirmed that positive attitudes to digital technologies 
reduce throughout training programmes (Wittenberg et al., 2021, p. 4). Overall, the 
triangulation of focus groups with student-teachers gathered compelling evidence 
into the student-teacher mindset to CALL. 
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1. 


Collaborative learning is one of the major tasks in language learning. The 
teachers have conducted collaborative learning tasks for decades, and recent ICT 


Abstract. This study investigated L2 learners’ perception changes at each stage 
of online collaborative writing. Previous studies revealed the familiarity of 
L2 collaborative learning with Information and Communication Technology 
(ICT), whereas few described at which stage of the learning process L2 learners’ 
perceptions change. Therefore, this study examines how the learners’ attitudes and 
perceptions change at certain phases of collaborative learning and whether these 
changes affect the success or failure of their L2 collaborative learning. This study 
analyzed two questionnaire surveys before and after the learning activity, observed 
the collaborative learning processes via visualization tools, and conducted semi- 
structured interviews for participants to reflect on their learning processes and 
perceptions of collaborative writing. The mixed research analyses demonstrate 
that advancing a particular stage leads to the learners’ linguistic awareness and the 
shift of their attitudes more positively. The findings show the factors and stages 
determining the success of L2 online collaborative learning. 


Keywords: collaborative writing, mixed methods research, data visualization, 


language awareness. 
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development and online instructions have boosted online collaborative writing and 
research (Arnold, Ducate, & Kost, 2012; Kessler & Bikowski, 2010; Lund, 2008; 
Yim & Warschauer, 2017). However, learners’ involvement during a collaborative 
task remains unclear. This study investigated the shift in L2 learners’ perception of 
online collaborative writing by focusing on the phases of the collaborative writing 
process. 


2. Method 


Our research recruited 98 undergraduate students in two Japanese universities 
specialized in economics (n=38) and engineering (n=60). Their L2 competencies 
are estimated to be lower-intermediate (A2) to intermediate (B1). 


The participants were asked to collaboratively write an essay consisting of several 
paragraphs using Google Docs for two weeks outside the classroom. They shared 
the same documents on their computer, decided their roles in their classes, and 
wrote the essay individually and in parallel. Both groups of university students 
conducted the same activities, while the topics given were slightly different; in one 
university, the participants were asked to translate the essays their instructor wrote 
in Japanese into English. The other participants from a science university, on the 
other hand, chose the topic by themselves and wrote their essays in English. 


This study employed a mixed method research, integrating a quantitative 
questionnaire survey and qualitative interview research. First, we conducted the 
questionnaire survey before and after the learning period. The questionnaire was 
developed to examine the perceived usefulness, attitudes toward, and reluctance to 
collaborative learning. Then one participant from each group was asked to join a 
semi-structured interview. Each participant was asked to review and elaborate their 
writing process and their thoughts on the collaboration. 


To observe participants’ collaborative writing processes, we utilized two online 
tools on Google Docs: DocuViz (Wang et al., 2014) and AuthorViz (Wang et 
al., 2018; Yim & Warschauer, 2017). DocuViz counts the number of words 
the participants wrote and edited during their writing period. AuthorViz can 
detect the author who wrote a specific part of the collaborative essay. These 
tools help the examinees to observe the collaborative writing process. As for 
the questionnaire, we analyzed the survey results by dividing them into positive 
and negative responses. Interviews were coded by extracting relevant items and 
conceptualized using grounded theory. Finally, we examined the relationship 
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between the questionnaire and interviews. The results of our quantitative and 
qualitative research are described in the next section. 


3. Results and discussion 


Figure | shows how one group conducts the collaborative task. The number of 
characters, or location of words in the essay was painted with each participant’s 
color from top to bottom. As seen in Figure |, the progress of the collaborative 
writing is visualized according to the timeline of writing. All participants joined 
the writing task one by one in Figure |, which suggests this group planned 
the task well and successfully conducted the collaborative task with equal 
contributions. Not every group shows a similar pattern. Only a few members 
joined the task in some groups, or the contribution of each member is only 
once in another group. The overall progress can be grasped when used with a 
visualization tool. 


Figure 1. Docuviz visualization for how one group develops their collaborative 
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The questionnaire items in Figure 2 are as follows: usefulness for collaborative 
writing, such as improvement of grammar, contents, or organization (Q1-Q3); 
attitude to collaborative writing (Q4); preference to collaborative writing (Q5); and 
resistance of interaction with others (Q5-8). As for the questionnaire, the results of 
QI to Q4 decreased. Meanwhile, the results of Q5 did not change. The results of 
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their resistance (Q5-8) differed — the resistance to being read or revised by peers 
decreased, but the sense of inequality in contribution increased. 


Figure 2. Positive responses (N1=75, N2=76) 
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Figure 3. Analysis of interview based on grounded theory 
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The qualitative result of the interview in Figure 3 above shows how participants’ 
perception changes in each stage of collaboration, i.e. brainstorming, collaboration, 


Tension Noticing 


Responsibility Habits 


No revision 
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and revision phases. In the brainstorming phase, participants had positive 
or negative perceptions of collaborative learning. Participants with positive 
perceptions noticed the advantage of collaboration. This advantage affected their 
collaboration through the subsequent two phases. On the other hand, participants 
with negative perceptions only talked about the task and did not improve their tasks 
or collaborative processes anymore. Their negative perception is due to member 
relationships, lack of time, and uneven contribution to the task. 


When participants were in the collaboration phase, participants who were positive 
toward collaboration connected their perception to the learning effect. Their positive 
beliefs were that (1) it was good to read other participants’ sentences; (2) it would 
improve their proficiency when they showed their writing to others; (3) they are 
getting accustomed to the collaborative task; or (4) it is good to hear advice from 
many other perspectives. Those beliefs all constituted their learning effect. Starting 
from this learning effect, three aspects of tension, noticing, and language awareness 
were repeated through collaborative learning and revision phases. Tension, 
consisting of audience, evaluation, and responsibility, would enhance participants’ 
concentration. Debski (2006) claims an audience can be L2 learning motivation 
because they provide immediate support. Noticing has a somewhat broad meaning. 
Participants noticed new or different ways of writing, found mistakes objectively, 
recognized the lack of grammar or phrases, and obtained different lines of thought. 
Those noticing aspects led to the positive participants’ language awareness in the 
revision phase. Participants could recognize their own or other participants’ writing 
styles and choice of words, which increased their writing variation and urged 
self-reflection on their learning. Participants became aware of the organization 
of an essay, self-revision, syntax, and dictionary use. The more participants were 
accustomed to the task, the more they were conscious of language. The language 
awareness increases the positive impression of collaboration on them and improves 
their learning. If participants did not revise their essays, they could not improve 
their collaboration. 


Although some participants did not resist collaboration from the beginning, 
some did. Some participants lacked self-confidence in their English proficiency. 
Others were reluctant to edit other members’ sentences, partly because they had 
less confidence in their proficiency, did not know how to revise, or were afraid 
of revising their peers’ drafts. That resistance, however, in many cases changed 
into their acceptance of editing. Some participants thanked other members for 
revising their writing, whereas others thought revision made them realize their 
lack of competency. In general, participants got accustomed to collaboration and 
decreased their resistance. 
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The questionnaire results suggest that the perception of usefulness and 
effectiveness decreases as a whole, because not all students have good impressions 
of collaborative learning. The data visualization and interview results, however, 
revealed that those with positive perceptions could obtain the learning effect of 
tension, noticing, and language awareness. Their positive perception could provide 
them with a good opportunity for collaboration, learning, and self-reflection. 


4. Conclusions 


Our mixed method study verified the shift in learners’ perceptions toward L2 
collaborative writing and examined how learners’ awareness changes through the 
phases of the writing process that lead to successful collaborative writing. As a 
result of our analyses, when learners have a positive perception of collaboration, 
they are able to develop their language awareness gradually across each phase, 
leading to successful learning. Additionally, our research confirmed the usefulness 
of the data visualization tools in clarifying the direction of the analyses. Therefore, 
more mixed method research will be conducted for technology-enhanced L2 
learning and teaching research using such visualization tools. 
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Abstract. The abrupt change from face-to-face to Online Learning (OL) in the 
emergency COVID-19 semester surprised and forced students to alter their study 
habits. Then came the second online period, and students were expected to be happier 
and more successful since now they were familiar with OL. Was this the case? Had 
the ways students learned, their perceptions of human interactions among teachers 
and students in OL, their opinions on the learning environment and their computer 
literacy changed? Our paper aims to answer those questions using comparative 
analyses of data sets from the first and second OL periods and attempts to uncover 
the positive and negative shifts and the topics that remained unchanged. The study’s 
findings show that COVID-19 related educational changes had multidirectional 
influences on students’ learning, ingroup interactions, and views about education 
and OL. Hopefully, the empirical data collected in this study will provide valuable 
information about OL’s immediate and prolonged effects. 
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1. Middle East Technical University, Ankara, Tiirkiye; ciler@metu.edu.tr; https://orcid.org/0000-0002-7171-1673 

2. University of Warsaw, Warsaw, Poland; e.gajek@uw.edu.pl; https://orcid.org/0000-0003-2971-8334 

3. University of Tuzla, Tuzla, Bosnia and Herzegovina; nihada.delibegovic@untz.ba; https://orcid.org/0000-0002-8241-6338 
4. University of Information Science and Technology “St. Paul the Apostle”, Ohrid, Republic of North Macedonia; lina. 
miloshevska@uist.edu.mk; https://orcid.org/0000-0003-3856-5448 


How to cite this article: Hatipoglu, C., Gajek, E., Delibegovié Dzani¢, N., & Milosevska, L. (2022). Comparative analysis 
of students’ views of online learning in the first and second Covid-19 semesters: examples from Tiirkiye, Poland, Republic of 
North Macedonia, and Bosnia and Herzegovina. In B. Arnbjérnsdottir, B. Bédi, L. Bradley, K. Fridriksdottir, H. Gardarsdottir, 
S. Thouésny, & M. J. Whelpton (Eds), /ntelligent CALL, granular systems, and learner data: short papers from EUROCALL 
2022 (pp. 154-161). Research-publishing.net. https://doi.org/10.14705/rpnet.2022.61.1451 


154 © 2022 Giler Hatipoglu, Elzbieta Gajek, Nihada Delibegovié Dzanié, and Lina Milosevska (CC BY) 


Comparative analysis of students’ views of online learning... 


“7 Introduction 


The first COVID-19 period of OL and teaching (2019-2020 spring semester) was 
characterised by the fast and stressful adaptation of teachers and learners to the 
new environment. To provide the best education they could, teachers learned how 
to use digital technologies and modify their teaching strategies (Hatipoglu et al., 
2021). Students acclimated to the new learning context without face-to-face contact 
with their teachers and peers and without ongoing support from their institutions 
(Miloshevska et al., 2020). 


The second semester of online education (academic year 2020-2021) was 
characterised by a higher level of customisation to the prolonged online education 
in countries such as Australia, the USA, Canada (Hickling et al., 2021), and Latvia 
(Baranova, Kobicheva, & Tokareva, 2021). However, the OL environments in 
Tiirkiye (TUR), Poland (POL), Republic of North Macedonia (RNM), and Bosnia 
and Herzegovina (B&H) have been largely unexplored. The present study aims to 
uncover the specific similarities and/or differences in students’ perceptions of the 
processes taking place in the two online teaching periods. It compares the results 
of surveys administered in TUR, POL, RNM, and B&H among university students. 


The research questions were: how do English language learners with diverse 
cultural and language backgrounds in TUR, POL, RNM, and B&H formulate their 
opinions on: 


* interactions among teachers and students; 

¢ methods utilised by students for learning; 

¢ difficulties students faced; 

¢ students’ opinions regarding the OL environment; and 
¢ — students’ evaluation of their computer literacy. 


2. Method 


2.1. Data collection 


The first dataset was collected in May-June 2020 (henceforth, the first period — 
FP) and the second in December 2020-January 2021 (hereafter, the second period 
— SP). To ensure parallelism among the datasets collected in TUR, POL, RNM, 
and B&H, a cross-culturally appropriate web-based questionnaire in English was 
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designed specifically for this study. The questionnaire, which included checkbox, 
Likert scale, and open-ended items, was used in both periods. 


2.2. Participants 
The number of participants in both periods was similar to each other (FP=216, 
of which Males=40.2%, Females=59.2%; and SP=214, of which Males=36%, 


Females=61.7%) and their country distributions were as shown in Figure |. 


Figure 1. Participants 
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2.3. Data analysis 


Two sets of descriptive analyses were performed on the data to demonstrate the 
common tendencies and the peculiarities appearing locally: across periods and 
countries. Due to word number restrictions, this paper presents across-period 
analyses with combined country results. 


3. Results and discussion 


3.1. Interactions among teachers and students 


Figure 2 shows that there are both similarities and differences between the 
examined periods. The similarity is that students interacted orally and in writing 
the most with their lecturers in both online periods. The key differences are 
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observed in the levels of interaction between individual students and student 
groups. The oral interaction between individual students increased 1.6 times, and 
group interactions 1.3 times in SP. Written interaction ‘within a group of peers’ 
also increased 1.2 times in SP. With these findings, we can argue that students’ 
autonomy and dependence on each other increased in SP. 


Figure 2. Similarities and differences between the examined periods 
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3.2. Methods utilised by students for learning 


Students’ autonomy and dependence on each other increased in SP of online 
teaching as seen from the data presented in Figure 3. Individual and collaborative 
work among students improved considerably, and 18% more students stated that 
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they worked collaboratively with a group of peers (Statement iii), and 7.3% said 
they worked together with a peer (Statement ii) to learn the material. 


The percentage of the students who completed their homework after synchronous 
classes increased by 15.4% (Statement v). The only less frequent activity in SP was 


Statement vi. 


Figure 3. Students’ autonomy and dependence on each other: ‘how do you study?’ 
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Possibly these increases are because students were more used to online teaching in 
SP and knew the importance of working individually and doing their homework to 
master the material. 


3.3. Difficulties students faced 


Similar to Kim and Asbury (2020), students reported facing difficulties in the 
following areas. 


¢ Technical problems. More students experienced technical problems such 
as unstable internet connection and limited access to the internet in SP 
(FP=65.3%, SP=81.8%). Also, a higher percentage had limited access to 
technology and devices (FP=15.3%, SP=19.6%). 


¢ Psychological problems. The number of students who stated they 
faced psychological problems such as ‘lack of motivation’ (FP=67.1%, 
SP=73.4%), ‘inability to concentrate because of boring classes’ (FP=44.9%, 
SP=55.1%), and ‘lack of organisation of their part’ (FP=35.6%, SP=39.7%) 
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increased in SP. A higher percentage of students also reported being 
overwhelmed by the number of online classes (FP=58.8%, SP=69.6%). 


¢ Institutional problems. The number of students confused by lack of 
organisation (FP=35.6%, SP=39.2%) and support from the institution 
(FP=27.8%, SP=36.5%) also increased in SP. 


¢ Instructional problems. More students reported a lack of support from 
the teachers (FP=34.3%, SP=43.9%) in SP. 


3.4. | Students’ opinions regarding the OL environment 

Students’ views related to OL were elicited via five-point Likert scale questions 
(five=strongly agree, one=strongly disagree), and a comparison of the mean 
scores related to the items showed a negative shift pertaining to all of them 


(Figure 4). 


Figure 4. Students views about OL 
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The means do not provide detailed insights into the specificity and backgrounds of 
each country’s changes. However, they illustrate well the general tendencies and 
reveal that students became more critical, neutral, or indifferent to OL in SP. 


Five-point Likert scale questions were also employed to reveal students’ opinions 
about OL. Similar to the set described above, the results showed a more negative 
evaluation by the students of OL in SP except for items iii and vi, where they 
disagreed less that ‘OL is useless’ and that they are ‘against OL’ (Figure 5). 
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Figure 5. Students’ opinions about OL 
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3.5. | Students’ evaluations of their computer literacy 


When asked to evaluate their level of computer literacy in FP, 54.9% of the 
students identified it as ‘Advanced’ and 41.6% as ‘Intermediate’. Almost opposite 
distribution was determined in SP. Only 40.3% classified their computer literacy as 
‘Advanced’ while 56.4% chose ‘Intermediate’ in SP. 


4. Conclusions 


The findings show that COVID-19 related educational changes influenced 
students’ learning to a great extent (cf. Cesco et al., 2021). However, the changes 
are multidirectional. On the one hand, the students were more teacher-dependent, 
but they also collaborated with individual peers and a group of peers more often, 
both orally and in writing, in SP. A holistic look at the difficulties and opinions they 
expressed in both periods allows for various interpretations. The psychological 
problems they observed might influence the other aspects of learning. With a lack 
of motivation, concentration, and self-organisation, all problems might grow, 
increasing the feeling of ‘supportlessness’, and decreased self-confidence even 
in their computer literacy. As all opinions tended to be either more neutral or 
critical in SP, we may infer that the students became more pessimistic, irreflective, 
or indifferent towards the learning environment they experienced. On-campus 
experience and a connection with universities are things that students missed 
during the online FP and SP. The study’s findings suggest that in universities, in- 
class instructions, including video-recorded ones, should be more common in the 
future than instruction solely available online. 
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Abstract. Foreign language teaching achieves best learning outcomes when 
individual differences of learners are taken into account. While it is difficult for 
teachers to support internal differentiation in the classroom, digital tools can 
adaptively propose individual learning paths through activities so that students can 
practice with appropriately challenging exercises. But how can sufficiently varied, 
systematically parametrized exercises be provided to enable a system to match them 
to individual learner needs? We present an approach for high-variability exercise 
generation that transforms a single specification into a multitude of exercises 
varying in complexity. The approach is currently being evaluated in a randomized 
controlled study in regular German seventh grade English classes, facilitating 
a systematic empirical exploration of adaptive exercise complexity in relation to 


learning outcomes. 
Keywords: intelligent language tutoring systems, ICALL, exercise generation, 
macro-adaptivity. 

A. Introduction 

Language instruction is challenged by the need to accommodate heterogeneous 


groups of students who differ in optimal learning paths and required support to 
achieve the best possible learning outcomes. Individual differences play a major role 
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in second language learning with students differing in a number of characteristics, 
including motivation, background, general language aptitude, and cognitive 
resources (D6rnyei, 2005). Yet instructors struggle with internal differentiation 
in classroom teaching as they generally lack time, suitable diagnostics, and 
appropriately adaptive materials. To speed up learning and improve learning 
outcomes, Intelligent Language Tutoring Systems (ILTS) aim to overcome these 
challenges by matching exercise properties to learner characteristics in an attempt 
to identify the best practice material at any time, for any student. Considering the 
vast space of exercise characteristics and combinations thereof, the availability of 
systematically parametrized learning material constitutes a major bottleneck for 
learner-adaptive digital systems. 


A number of ILTS have been developed for foreign language learning. Examples 
successfully used in real life include Heift’s (2010) E-Tutor, Nagata’s (2009) 
Robo-Sensei, and Amaral and Meurers’s (2011) Tagarela. Such systems can in 
principle support two types of adaptivity, micro- and macro-adaptivity. Most 
systems focus on providing scaffolding feedback that successively guides a 
learner toward the correct answer, i.e. micro-adaptivity. Adaptive exercise 
sequencing attempting to always select activities in the learner’s zone of 
proximal development has received less attention. The challenge with such 
macro-adaptivity consists in a shortage of different activities which, if overcome, 
would allow adaptive systems to provide exercises whose nature and complexity 
matches a learner’s characteristics and proficiency. Manually creating all these 
exercises quickly becomes unfeasible considering the number of possible 
combinations of exercise parameter settings. A systematic approach, however, 
could generate a multitude of activities from a single, well-defined specification. 
To this purpose, we present an approach for high-variability exercise generation 
with the goal of efficiently generating practice material for user-adaptive ILTS 
targeting beginning to intermediate learners. 


2. System description 


The approach focuses on exercises practicing grammatical concepts from the 
official English curriculum in Germany. The language-aware search engine 
FLAIR (Chinkina & Meurers, 2016) assists users in identifying texts rich in 
pedagogically relevant linguistic structures. Although, at this point, the high- 
variability exercise generation requires manually compiled specifications, FLAIR 
thus provides a natural system context for an exercise generation functionality. 
To use the new exercise generation in FLAIR, users upload a specification file 
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in Excel or JSON format and select the targeted grammatical concept and the 
output file format. The algorithm generates abstract exercise definitions from the 
specification for the entire space of pre-defined parameterizations. Parameters 
influencing exercise complexity include: (1) the exercise type, including 
memory, multiple choice, underline, jumbled sentences, categorization, and fill- 
in-the-blanks; (2) parameters specific to exercise types, such as the number and 
type of distractors for multiple choice activities or the nature and position of 
hints for fill-in-the-blanks exercises, e.g. lemmas above the exercise or behind 
the gaps; and (3) parameters specific to a grammatical concept, e.g. which clause 
is turned into the relative clause of a relative sentence, clause order in conditional 
sentences, or whether the exercise focuses on practicing verb forms in if-clauses 
or in main clauses. 


Figure | takes relative clauses as an example and outlines how different exercise 
types make use of different specification components for exercise generation. 
Memory activities, for example, use the information highlighted in bold font: the 
chunks of both clauses and the relative pronoun. 


Figure 1. Parameter usage of different relative clause exercises 
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Pronoun clause 2 


Jumbled 
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Figure 2 below illustrates how this subset of specification components is used 
to generate four distinct memory exercises for relative clauses. The parameters 
responsible for this variability consist in the order of the main clauses as well as in 
the choice of clause used for the relative clause in the relative sentence. 


Depending on the selected output format, the algorithm transforms the generated 
abstract exercise definition into HSP files, which can be uploaded to any learner 
management system supporting the standard, or a proprietary XML format 
compatible with the FeedBook ILTS (Rudzewitz et al., 2018). 
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Figure 2. Generation of Relative clause Memory exercises from a specification 
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3. Evaluation and discussion 


The approach targets grammatical concepts from the seventh grade English 
curriculum in Germany. The current implementation covers relative clauses with 
focus on pronoun usage or contact clauses, and conditionals practicing verb forms 
of a given conditional type or the differentiation between conditional types. The 
specifications for different foci on relative clauses differ slightly due to differing 
requirements of the resulting exercises. The specifications for conditionals can 
in principle be re-used if the grouping of specification items is altered in such a 
way that all items of a group belong to the same or to different conditional types. 
For relative pronoun practice, the algorithm supports automatic generation of 
19 exercises from a single specification. Contact clauses yield three different 
exercises. For verbs of conditional sentences, the algorithm generates 48 exercises 
per specification. Exercises practicing the identification of conditional types come in 
39 variants. This promising space of exercises is currently being empirically tested 
in a project related to the FeedBook focused on adaptive sequencing. In line with the 
setup advocated in Meurers et al. (2019), we are currently conducting a year-long 
randomized controlled field study in regular English classes in secondary schools. 


Limitations of the current approach include the handling of discourse phenomena 
such as coreference and cohesion. Example 1 illustrates that for conditional 
sentences, the semantic validity is challenged if the first clause contains pronouns 
referring to an antecedent in the second clause. Also related to clause order, 
Example 2 illustrates how cohesion can become an issue with de-contextualized 
relative clauses if the clauses have a natural order which is violated by the generated 
relative sentence. An additional limitation is introduced by the naive nature of 
the exercise generation algorithm which cannot resolve co-references unless 
the surface forms are identical, so that Example 3a cannot be transformed into a 
relative sentence but instead requires a specification such as given in Example 3a’. 
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(1) a. If Tommy found his glasses, he would wear them. 
b. # He would wear them if Tommy found his glasses. 
b’. Tommy would wear his glasses if he found them. 


(2) a. The boy who came in greeted everyone. 
b. # The boy who greeted everyone came in. 


(3) a. The boy came in. He greeted everyone. 
a’. The boy came in. The boy greeted everyone. 


We are therefore currently introducing further natural language processing 
analyses to overcome these limitations and to make it possible to add parameters 
that are characteristic of developmental sequences discussed in the second 
language acquisition literature (e.g. Pienemann & Johnston, 1986), such as the 
use of negation or interrogative structures. In addition, we are implementing 
a user interface to facilitate specification authoring and to avoid formatting 
issues of the input files. This interface will also merge different specifications 
of a grammatical concept and thus allow even greater exercise variability per 
specification. Since the exercise generation is already embedded into a language- 
aware search engine, another strand of current work focuses on the generation of 
exercises from authentic texts. 


4. Conclusions 


We have presented an approach to support macro-adaptivity in ILTS through high- 
variability exercise generation. In future work, we plan to extend the approach to 
a broader range of grammatical concepts. We also intend to evaluate the influence 
of different parameters and their interaction on exercise difficulty, approximated 
by student performance. To this purpose, we hope to gain first insights from 
the ongoing field study which we will then broaden through analyses based on 
controlled lab studies on exercises with systematically manipulated parameters. 
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Abstract. We propose to use reading time as a metric to report progress in language 
learning applications. As a case study we use a web-based application that enables 
learners of a foreign language to read texts from the web and practice vocabulary 
with interactive exercises generated based on their past readings. The application 
captures generic interactions with the web page (e.g. switching to a different tab) 
but also interactions directly related to language learning (e.g. clicking on a word to 
get a translation). We propose two metrics for approximating reading times based 
on user interactions with the web application. We analyze the correlation between 
these metrics and other interaction metrics and show that active time is the best 
metric for estimating the user’s actual involvement with the texts and that it can be 


approximated from interaction metrics. 


Keywords: reading time, second language learning, web-based learning, user 


interactions. 


ale Introduction 


We propose to use reading time as a metric to report progress to learners and 
teachers in language learning applications and in particular, in applications that 
support extensive reading of authentic texts in L2. Given that reading time is a 
good predictor of learning (Wallot et al., 2014), one simple but meaningful way 
to provide feedback to the readers is to show them the time they spend reading in 
the language of their choice. Measuring the reading time in the wild, as opposed to 
measuring it in controlled experiments in the lab, is however challenging: one must 
handle the fact that the learner behavior can only be estimated but not fully known. 
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As a case study, we use a web-based application named Zeeguu’; as mentioned in 
Lungu, van den Brand, Chirtoaca, and Avagyan (2018), this web-based application 
enables learners of a foreign language to read texts from the web about topics they 
are interested in and practice vocabulary with interactive exercises based on their 
past readings (Lungu & van Engen, 2019; Lungu et al., 2018). 


We deployed this application in the wild for more than one year and for more than 
606 users. The non-controlled environment means multiple challenges to reading 
speed estimation, e.g. users might start reading an article, but then go to make 
a coffee, or switch to a different tab to check social media. Sometimes readers 
will open an article, scroll through it for a bit, and then close it because it is not 
interesting to them. 


To be able to estimate reading time, the web application uses telemetry regarding a 
variety of user interactions. These interactions include: when an article is opened and 
closed, when the browser tab is out of focus, and recording when the user provides 
feedback on the article. The application also captures interactions directly related 
to language learning, e.g. clicks on words and phrases for getting translations and 
pronunciations. Finally, the application also presents the user information about 
their reading time with the help of an activity dashboard. 


2. Method 


2.1. Data 


The data used for analysis can be found in the July 2022 Zeeguu Anonymized DB 
Dump‘. The analysis is based on data from 606 highschool and university students 
from the Netherlands and adult learners from several language schools in Denmark 
who were studying French, Dutch, and Danish as foreign language. Their data can 
be found as CSV files in the reading-times--eurocall22 folder at https://github. 
com/zeeguu/studies. 


2.2. Reading time approximation 


Eye tracking technology would provide a perfect measurement of reading time 
(e.g. Hollenstein, Barrett, & Bjérnsdottir, 2022), but in its absence, we have to use 
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approximations based on user interactions. Based on the collected interaction data 


we evaluate two alternative ways. 


¢ Apparent time: the full duration between opening and closing a text, 
used in past works not specific to language learning (Hwang, Tsai, Tsai, & 


Tseng, 2008; Sheard, Ceddia, Hurst, & Tuovinen, 2003). 


¢ Active time: the result of subtracting from apparent time (1) out of focus 
time — time spent on other windows or tabs and, (2) idle time — time 
when a user does not interact with the system for more than a minimum 


threshold>. 


Figure | shows all the events that are recorded from the interaction of the user with 
ID 3083 (from the DB dump mentioned above) with the article with ID 1973640. 


Figure 1. An example of a very large difference between Apparent Time (26min) 


and Active Time (4min) 
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3. Reading time versus other metrics 


We investigate whether any metric derived from the events that we collect can 
be used to predict reading time. This would be useful for estimating reading time 
when there is no previous user data about a given article. 


We use Spearman’s correlation to measure the strength and direction of the 
association between two variables computed from user interactions. These reading 
time measures are averaged across all events/readers for a given article. We 
consider the following metrics for each article: 


* word count, the number of words in a given article; 
¢ liked, the number of times the article was liked by readers; 


¢ translated, the number of words that were clicked to be translated in this 
article; 


¢ spoken, the number of words that were clicked to be pronounced in this 
article; 


* opened and closed, the number of times a given article was opened or 
closed; 


¢ difficulty, the (between | and 100) normalized Flesch Kincaid reading 
score (higher=easier to read), as well as perceived difficulty (easy/ok/hard, 
the user feedback about their perceived difficulty after reading an article); 


* time_to_first translation, the time before the first translation occurs, 1.e. 
the duration between opening an article and the first translation. 


Table | presents the correlation coefficients p between these variables across all 
articles of all languages. Note that the correlation values are generally moderate, 
since the data is sparse, i.e. many fields are empty because it is optional for the 
user to give feedback or like an article. These results show that text difficulty does 
not correlate with reading times, presumably because readers automatically choose 
articles they feel confident they can finish in reasonable time. However, the number 
of words a user clicks to get a translation or pronunciation correlates with text 
difficulty suggesting that the used difficulty metric is a good predictor of actual 
perceived difficulty. 
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The strongest correlation in Table | is seen between word count and reading time 
(both the reading time metrics in this case). However, ACTIVE TIME correlates 
most strongly with word count and text difficulty suggesting that it is a better 
measure for estimating the actual time the users are involved with the text. 


The results also show that users are more likely to like an article if they read it 
faster and need less translations. This hints at the importance of ensuring that the 


articles are of the right difficulty for the users. 


Table 1. Correlation analysis 
varl var2 Spearman’s p 
word_count difficulty -0.05* 
word_count translated 0.20* 
word_count APPARENT_TIME 0.42* 
word_count ACTIVE TIME 0.42* 
word_count time_to_first_translation 0.26* 
difficulty liked 0.05* 
difficulty translated -0.18* 
difficulty spoken 0.17* 
difficulty opened -0.19* 
difficulty closed -0.18* 
difficulty APPARENT_TIME -0.06* 
difficulty ACTIVE_TIME -0.07* 
difficulty time_to_first_translation -0.04* 
liked translated -0.11* 
liked APPARENT_TIME -0.03* 
liked ACTIVE_TIME 0.06* 
liked time_to_first_translation -0.10* 
translated spoken 0.27* 
translated APPARENT TIME 0.39* 
translated ACTIVE_ TIME 0.42* 
translated time_to_first_translation 0.81* 
spoken APPARENT_TIME 0.30* 
spoken ACTIVE_TIME 0.27* 
spoken time_to_first_translation 0.44* 
APPARENT_TIME ACTIVE_TIME 0.84* 
time_to_first_translation APPARENT TIME 0,59* 
time_to_first_translation ACTIVE_TIME 0.59* 


Significant results (p<0.01) marked with * 
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4. Conclusions and future work 


Our results suggest that active reading time is a better metric than apparent reading 
time and that it is possible to approximate reading time from both text-based and 
user-derived metrics. In the future, we plan to investigate personalizing the reading 
time estimation by applying prediction models based on a single user’s past text 
interaction. We also plan to investigate presenting to a user reading times that 
other users required for a given article, in the same way in which sports tracking 
applications report comparable users achievements (West, 2015). 
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Using an online high-variability phonetic 
training program to develop L2 learners’ 
perception of English fricatives 


Atsushi Iino! and Brian Wistner? 


Abstract. This study investigated the degree to which Japanese learners of 
English accurately perceive English fricatives over time and the extent to which 
fricatives were misidentified. To train and measure perception skills, an online 
high-variability phonetic training program was used in an English as a Foreign 
Language (EFL) class in Japan for eight weeks. The results indicated that learners’ 
perception of some of the fricatives improved over time, while others remained 
difficult to distinguish from other fricatives. Implications for EFL pronunciation 


instruction are considered. 


Keywords: pronunciation, perception, fricatives, high-variability phonetic training. 


1. Introduction 


For Japanese learners of EFL, pronouncing English fricatives such as [f], [v], [@], 
and [6] can be challenging (Fujinuma & Wilson, 2010; Kawasaki & Tanaka, 2012; 
Lambacher, Martens, Nelson, & Berman, 2001). Flege’s (1995) speech learning 
model predicts that second language (L2) sounds that are similar to first language 
(L1) sounds are relatively harder to master, while new sounds that are different 
from L1 sounds are relatively easier. It has often been observed that L1 Japanese 
speakers replace the English dental fricative /0/ with the fricative /s/, and /v/ with 
/b/, such as in Sank you bery much (Thank you very much). Such replacements 
may indicate that these English sounds are challenging to articulate for some 
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EFL learners. One question that arises from this observation is whether learners 
distinctively perceive the fricative sounds or misidentify them at the perceptual 
level. If misidentification is involved to a certain degree, learners’ perception 
of English fricatives and the sounds used to replace them may be the cause of 
confusion in production. 


L2 learners are expected to be able to perceive not only standard English sounds but 
also variations in the same phonemes. This ability is of increasing importance as 
English is used as a lingua franca throughout the world. High-Variability Phonetic 
Training (HVPT) enables L2 learners to practice perceiving L2 sounds produced 
by a variety of talkers in various phonetic environments (Thomson, 2012) and 
has been shown to improve L2 lexical processing (Melnik & Peperkamp, 2021). 
HVPT can be provided in an intensive computer-assisted environment so that 
learners can check their perception of the sounds in a self-directed manner inside 
and outside the classroom. The purpose of the present study was to examine the 
effects of computer based HVPT on L2 learners’ perception of English fricatives. 
The research questions were as follows. 


¢ To what extent do Japanese learners of English accurately perceive English 
fricatives? 


¢ To what extent are the target English fricatives misidentified as other 
English fricatives? 


2. Method 


Participants were five university sophomores whose L1 was Japanese. They were 
enrolled in a mandatory English course in the Faculty of Economics in a university 
in Tokyo. They used English Accent Coach (EAC; Thomson, 2017), which is an 
online program that provides an HVPT environment to practice perceiving the 
following English fricatives: [f], [v], [8], [6], [f], and [z]. In addition to those, a 
bilabial [b] was also added as a target sound because Japanese speakers tend to 
replace [v] with [b]. 


The participants were assigned to use EAC once a week in class for perception 
training for eight weeks in a time-series design. Within EAC, the learners were 
directed to set the linguistic environment to ‘initial consonant + all vowels’, 
which means non-word single syllables were used as stimuli. They then began the 
forced-choice identification task with 100 stimuli in the high-variability condition 
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where stimulus sounds are provided randomly from differently accented talkers. 
The participants were assigned to complete a special worksheet created by the 
researchers in which they recorded misidentified sounds every time they failed to 
correctly perceive the stimuli. 


3. Results and discussion 


The mean accuracy rates at the end of the eight weeks for each target sound 
indicated that the lowest ones were 43.0% for [v], followed by 50.2% for [0], 
57.2% for [z], and 61.3% for [6] (Table 1). The sound of [b] (80.1%), [f] (81.6%), 
and [f] (95.5%) showed relatively higher overall success rates. Regarding changes 
over time, [b] began at 64.9% in Week | and improved gradually to 92.4% in 
Week 8 (Figure 1). Progress was also observed for [0], from 43.5% in Week 1 
to 61.4% in Week 8. More HVPT opportunities or other consciousness-raising 
instructional tasks might be required for the sounds on which progress was not 
observed, while some might have exhibited a ceiling effect. 


Table 1. Progress in perception of target sounds over eight weeks (%) 


wi Ww2 Ww3 w4 WSs Wo W7 Ws M 

83.1 78.2 81.5 86.0 87.8 66.7 86.1 83.2 81.6 
43.2 43.3 49.5 47.5 42.9 33.3 38.9 45.7 43.0 
43.5 39.2 48.1 38.7 51.2 58.9 60.7 61.4 50.2 
63.5 52.8 65.0 60.8 67.7 64.7 56.6 59.4 61.3 
96.9 82.8 97.1 94.5 98.6 100 97.9 96.3 95.5 
59.1 54.6 59.0 66.6 60.5 45.9 59.4 52.6 57.2 
64.9 76.3 69.2 66.0 88.6 90.0 93.3 92.4 80.1 
64.9 61.0 67.1 65.7 71.0 65.6 70.4 64.9 67.0 
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Figure 1. Changes in perception over eight weeks (%) 
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Regarding the misidentification rates (Table 2, Figure 2), the most misidentified 
sound was [v] (56.3%), which was mistaken mainly as [b] (31.8%). This relationship 
was also reflected in the results for [b] (22%) to [v] (17.7%). The sound [v] was also 
mistaken as [6] (12.2%), which was mentioned in Fujinuma and Wilson (2010). 


The sound [0] (51.6%) was mostly confused with [f] (36.2%) and sometimes with 
[f] (8.2%). The sound [6] (38.5%) was most confused with [z] (15.7%) or [v] 
(15%). This relationship was also observed for [z] (42%) and [6] (35.4%), which 
was the most confusing sound. Finally, the sound [f] (17.9%) was mistaken most 
for [8] (10%), which was consistent with Lambacher et al. (2001). 


Table 2. Error and misidentification rates for each sound over eight weeks 


Misidentification rate (%) 

Target | No. of Error f v ti) fi) S Z b 
sounds | items (M) | rate (%) 

f 13.1 17.9 0 1.5 10 1.6 1.8 0.7 2.4 
Vv 13.2 56.3 4.2 0 2.6 12.2 |0 5.6 31.8 
0 11.5 51.6 36.2. 0.7 0 4 8.2 1.1 1.5 
6 14.5 38.5 1.8 15 0.4 0 0 15.7 |5.7 
S 13.8 4.9 0.9 0 2.8 1 0 0 0.2 
Z 16.1 42.0 0 0 0.2 41.8 0 0 0 

b 17.7 22.0 1.4 17.7 |0.4 2 0 0 0 


Figure 2. Details of misidentified sounds 
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These results indicate that the reciprocal relationship between [f] and [6] might 
have contributed to the improvement in perception of the dental fricative [6]: the 
most misidentified sound for [f] (17.9%) was [@] (10%), and the sound [8] was 
most misidentified as [f] (36.2%) (Table 2). A paired relationship was observed for 
[v] and [6], which means while the target sound [6] was mistaken for [v] (15%), 


177 


Atsushi lino and Brian Wistner 


the target [v] tended to be perceived as [6] (12.2%). Even though progress over 
time was not observed, reciprocity was also found. The case between [b] and [v] 
was also similar. The target sound [v] was often misidentified as [b] (31.8%) and 
[b] was mistaken as [v] (17.7%). Although the recognition of [v] was the lowest 
for the entire period, the perception rate of [b] improved the most. Additionally, [6] 
tended to be perceived as [z] (15.7%), and the target [z] tended to be misidentified 
as [6] (41.8%). From these data, the high functional load pairs elicited from HVPT 
might be the key features for developing learners’ consciousness of the acoustic 
differences during instruction, which could also facilitate production of the sounds. 


4. Conclusion 


The results of the current study indicated that weekly practice using HVPT could 
facilitate L2 learners’ perception of some English fricatives, and the misidentification 
rates helped to identify confusing pairs or sets of English fricatives: [6] and [f], [v] 
and [b], and [ 6] and [z] / [v]. Considering the limited time available for the in 
class training, further progress might have been observed if the learners had used 
EAC outside of class. Further research could examine such a combination as well 
as having learners repeatedly record and reflect on their perception and production 
accuracy as they utilize HVPT. 
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Strategies to build a community 
of learners in online classes 


Bradley Irwin! 


Abstract. This paper describes the use of screencast feedback to enhance teaching 
presence and weekly self-reflection surveys to enhance social presence among 
students and build a community of learners. These approaches were found to 
diminish the feeling of isolation in online learning environments. Further findings 
indicated that this multimodal approach to providing feedback helped build a 
rapport between the teacher and students which led to increased motivation and task 
engagement. These strategies to build a community of learners will be of particular 
interest to educators looking for innovative approaches to improving teaching and 
social presence in online learning environments while also providing personalized, 
formative feedback. 


Keywords: multimodal feedback, screencasts, online learning, teaching, social 
presence. 


1. Introduction 


At the outset of the coronavirus pandemic, Japanese universities experienced an 
unprecedented shift to online learning. Although Japan is often thought of as a 
leader in innovative technology, most classes at the tertiary level are still very 
much analog. To make matters worse, the decision to ‘pivot’ to online learning 
was made only weeks before the beginning of the school year at many institutions 
in April 2020. While studies by Kumar, Martin, Budhrani, and Ritzhaupt (2019) 
and Hodges et al. (2020) found that effective online university-level courses took 
between six to nine months of thoughtful planning, teachers in Japan had a relatively 
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short amount of time to design their courses. Furthermore, a lack of educational 
technology support led many analysts to predict that the Japanese education system 
would have a particularly difficult time to adapt to online learning environments 
(Kittaka, 2020). 


To better understand the results of this shift to online learning, the Ministry of 
Education, Culture, Sports, Science, and Technology (MEXT) conducted a survey 
of 3,000 randomly selected students in early 2021. While the survey found that 
almost 60% of respondents were satisfied or somewhat satisfied with their online 
classes, it also found that 53% felt a feeling of loneliness and isolation from their 
peers (MEXT, 2021). 


Furthermore, a study conducted by Irwin (2021) of 103 first- and second-year 
university students found that 65% of respondents felt isolated and socially distant 
from both their classmates and teachers. Students were missing what Lawrence 
(2017) described as ‘the human feel’ in their online classes caused by a lack of 
social and teaching presence. Lawrence identified social and teaching presence as 
essential for successful online learning, and that, “teaching presence is the catalyst 
that fosters social and cognitive presence” (Lawrence, 2017, p. 142). 


Therefore, the author of the present study developed two strategies to address 
the perceived lack of interpersonal interaction students reported experiencing in 
their online classes. The two instructional strategies described in this paper were 
developed to promote teaching presence by providing personalized screencast 
feedback and social presence through weekly self-reflection surveys. The goal of 
each was to help students establish a rapport with their teachers and classmates 
while also fostering a sense of community in their online lessons. 


2. Instructional learning context 


Two hundred and fifty first- and second-year undergraduate English as a foreign 
language learners (A2-B1 CEFR level) enrolled in ten online classes (divided by 
year and language ability) participated in the lessons described in this paper. The 
lessons were conducted online in English and held once a week for 90 minutes 
during a 15-week semester. The lessons combined real-time instruction using a 
video conferencing platform with on-demand materials in the form of video 
tutorials, narrated slideshow presentations, and study materials uploaded to a 
Learning Management System (LMS). The lessons and study were conducted 
during the 2021 spring term (April-July). 
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3. Personalized screencast feedback 


To improve teaching presence in the online lessons, the instructor utilized 
screencasts to provide personalized feedback on a variety of assignments. The 
screencasts were in the form of digital video recordings with audio narration that 
captured feedback on digitally submitted assignments displayed on a computer 
screen. 


Each screencast feedback video followed the format of feedback conferences 
outlined by Ferris and Hedgcock (2005). The screencasts started with a salutation 
and affirmation, highlighted the strengths of the assignment, discussed its 
weaknesses, provided suggestions for improvement, and finished with closing 
remarks of encouragement. Following this format, the screencast feedback was 
highly personalized and provided an emotive element conveyed through the tone 
and inflection of the instructor’s voice. 


The length of feedback videos and the time it takes to produce them varies depending 
on the type of assignment and feedback focus. As an example, for a paragraph 
writing assignment of 150 words, the videos averaged four minutes in length and 
took approximately 15 minutes to produce. The production time required depends 
on the screencaster experience, workflow efficacy, and teaching context. 


In terms of improved teaching presence, Irwin (2022) describes an experiment in 
which students were separated into a group that received written feedback (control) 
and a group that received screencast feedback (experimental). Among several 
significant findings, students in the experimental group reported that the feedback 
style resulted in enhanced feelings of encouragement and a closer connection 
with the instructor. Thus, screencast feedback helped develop a rapport with the 
instructor that did not evolve from traditional written feedback. 


4. Weekly self-reflection surveys 


To address social interaction, weekly self-reflection surveys were utilized. The 
surveys were conducted using an online form and were to be submitted by the end 
of each week. 


While the content of the surveys changed slightly each week depending on the 
lesson materials, each survey followed a similar pattern. Students were asked to 
report how they felt during the week, the amount of time they had spent engaging 
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with the lesson material, what they had learned, what they had trouble with, and 
if they had any comments or suggestions for the instructor to help improve the 
lessons. 


The survey results were then anonymized and compiled into a spreadsheet that 
was uploaded to the LMS and shared with the class. To acknowledge student 
contributions, the instructor highlighted responses in each question category that 
were deemed to be insightful or thought-provoking. Finally, the instructor created 
a short screencast of the survey results to encourage the students and respond to 
any difficulties they were having. This form of contextual socialization allowed 
students to compare their efforts, learning outcomes, and difficulties with their 
classmates. By acknowledging the student contributions in this way, a virtuous 
cycle of feedback and response was created. This was demonstrated by the fact that 
the thoughtfulness of the responses each week deepened and developed from the 
outset of the activity until the end of the term. 


5. Conclusions 


By combining the assignment screencast feedback and weekly self-reflection 
surveys, the instructor was able to enhance both teaching and social presence in 
the online lessons described in this paper. As a result, students reported extremely 
high satisfaction levels in anonymous course evaluation surveys conducted by 
the university at the end of the semester. Further results showed that students 
appreciated the instructor for utilizing the two strategies because they felt that they 
had developed a genuine rapport with both the teacher and their classmates. 


The challenge of planning and implementing online language lessons that provide 
appropriate teaching and social presence should not be underestimated. However, 
without sufficient attention given to interpersonal interaction, instructors risk 
increasing attrition rates in their online classes (Tyler-Smith, 2006). The two 
strategies described in this paper can go a long way to creating an online learning 
environment rich in both teaching and social presence while at the same time 
providing highly personalized and motivating feedback. 


It is difficult to predict what lasting impacts the coronavirus pandemic will have on 
education in general and online learning specifically. However, screencast feedback 
and weekly self-reflection surveys are not activities that should be limited to online 
learning environments. They can be used to create unique opportunities to develop 
and strengthen rapport within hybrid style or in-person lessons as well. 
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Abstract. This study examines whether ten weeks of direct instruction of affixes with 
online systems can improve learners’ knowledge of affixes by using Affix Master 
10 (AM10), a collection of online self-study materials developed by the current 
researchers. The aim of AM10 is to let students comprehensively learn 30 prefixes 
and 31 suffixes. The two research questions are (1) whether systematic instruction of 
affixes using the online program improves learners’ knowledge of affixes (prefixes 
and suffixes), and (2) whether systematic instruction of affixes using the online 
program improves learners’ knowledge of affixes without direct instruction as well 
as those with direct instruction. The results found that after learning with AM10, 
the affix knowledge statistically significantly increased. Furthermore, it seemed to 
contribute to the learning of the affix system itself, which was evidenced by the gain 
in knowledge of untaught affixes. 


Keywords: affix, direct instruction, online program. 


1. Introduction 


Expanding word knowledge through affixation is a familiar L2 learning strategy. 
Nagy and Anderson (1984) stated that “if the frequent words in a word family 
are already known, this procedure provides a bridge from familiar words to new 
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words” (p. 326). The concept of word families (i.e. a word stem and its collection 
of affixes) allows learners to infer the meaning of other members of the same 
word family if they know the affixed word (Bauer & Nation, 1993). 


However, there has been insufficient research on the knowledge of affixes among 
Japanese learners of English. Most studies on this topic have focused on the 
relationships between lexical knowledge and affixes, and between productive 
and receptive knowledge of affixes (Aizawa, Iso, & Nadasdy, 2019; Mochizuki & 
Aizawa, 2000; Schmitt & Meara, 1997). These studies show that the relationship 
between lexical knowledge and affix knowledge as well as between productive 
and receptive affixation knowledge is highly correlated. However, it has not been 
tested whether increase in affixation knowledge is caused by increased lexical 
knowledge or by increased knowledge of affixation itself. 


Schmitt and Meara (1997) found that without educational intervention, the learning 
of English affixes is limited, but there has been little systematic teaching of affixes 
in Japan. This is partly because the type and number of affixes presented to learners 
vary in different textbooks (Morita, Uchida, & Takahashi, 2019). Therefore, the 
aim of this study is to investigate the effect of systematic teaching of affixes using 
a collection of original materials, AM10, as well as its extended effect, if any, on 
untaught affixes. 


Our research questions are as follows. 


¢ Does systematic instruction of affixes using AM10 improve leamers’ 
knowledge of affixes between pre- and post-test? 


* Does AM10 improve learners’ knowledge of affixes without direct 
instruction as well as those with direct instruction? 


2. Method 


The affixes targeted for study were selected according to two criteria: (1) The 
stem of the word must be within the 4,000-frequency band in the JACET8000 
vocabulary list, and (2) affixes must have three or more different stems to be added 
to form new words. Finally, 30 prefixes and 31 suffixes were selected as in Table |. 
The procedure of the on-demand self-study program was made up of four steps as 
shown in Table 2 and Figure 1. 
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Table 1. Target affixes introduced in AM10 


Week 1 Prefix un-, in-, dis-, non- Negative 
Week 2 anti-, contra-, counter-, with- Against 
Week 3 inter-, pre-, post-, re- Before, forward, in, 
after, backward 
Week 4 super-, sur-, extra-, ultra-, Over, beyond, down, 
sub-, in-, en-, em- under, in, within 
Week 5 com-, CO-, cor-, syn-, sym-, With, number 
semi-, uni-, mono-, homo-, bi- 
Week 7 Suffix -an, -eer, -ee, -er, -ist Person 
Week 8 -al, -ance, -ency, -ion, Abstract noun 


-ment, -ism, -dom, 
-ness, -ship, -th 
Week 9 -ful, -ous, -y, -ish, -ern, Adjective 
-able, -ible, -less, 
-ate, -al, -ive 
Week 10 -1ze, -ify, -ate, -ly, -wards Verb, adverb 
Week 11 | Wrap-up | Review 


Figure 1. Components of AM10 


Affix Master 10 
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Step 1: explanation 


9 VOCALS Step 1: Watch a video with 
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2 A 2 j SHE D2 CASES Step 3: Do an exercise and 


submit a result. 


\ r) F | 
e be Step 4: Take a quiz with a 
id 


passing score of 80%. 


(Otherwise, go back to step 
1 or 2.) 


Table 2. Four stages of weekly sessions 


Activities Interface | Devices Duration 
Step 1 | Watching video WebClass | PC, Tablet, or Smartphone | 7-12 mins 
Step 2 | Additional examples | VOCALS | Smartphone 10-30 mins 
Step 3__| Exercises WebClass | PC 15-30 mins 
Step 4 | Taking a quiz WebClass | PC, Tablet, or Smartphone | 15-30 mins 
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To measure the learning outcome, an online affix test (based on Aizawa et al., 2019) 
was used. The task was to choose one out of eight affixes commonly attachable to 
the three stems. For example, when ‘clear’, ‘complete’, and ‘quiet’ are given as 
a cue, participants were to choose ‘-ly’ among ‘-al’, ‘-ment’, ‘-ist’, ‘-ous’, ‘-ful’, 
‘-ize’, ‘-ly’, and ‘-ation’. The total number of questions was 60 (15 prefixes and 
15 suffixes, two different sets of stems for each). 


Participants in this study were 512 Japanese university students majoring in 
engineering. Their proficiency levels are estimated as CEFR A2 to B1 level. They 
had at least six years of formal English education. 

3. Results and discussion 


Table 3 and Figure 2 show the results of the affix test at the pre- and post-tests. 


Table 3. Mean raw scores with standard deviations across time 


N Week 0 Week 6 Week 12 
Mean (S.D.) Mean (S.D.) Mean (S.D.) 
Combined 512 33.05 (10.72) 35.67 (12.09) 37.48 (12.28) 
Prefix 12.52 (5.49) 15.42 (6.22) 15.54 (6.43) 
Suffix 20.53 (6.12) 20.25 (6.72) 21.95 (6.62) 


Figure 2. Mean scores across time 
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The scores of the prefixes increased at Week 6, as did the scores of the suffixes 
at Week 12, which corresponds to the period when the respective types of affixes 
were learned. These gains were revealed as significant with ANOVA and a post- 
hoc comparison (F(2,1022)=77.55, p<.001, partial n*=0.132; t=-11.45, p<.001, 

=-0.506 for prefix, and F(2,1022)=33.95, p<.001, partial 7°=0.62; t=-6.41, 
p<.001, d=-0.283 for suffix). Further, there was no significant difference in the 
mean scores during Weeks 6 to 12 in prefix. This indicates that the affix knowledge 
gained through AM10 was retained after six weeks. Taken together, these results 
demonstrate the effectiveness of AM10. The results extend the findings of Schmitt 
and Meara (1997). 


Concerning the effects on affixes that were not taught in AM10, Figure 3 shows 
the changes in the amount of knowledge over time for the four affixes: ‘fore-’, 
‘-ity’, ‘-(e)n’, and ‘-ant’. Here, the perfect score for each affix indicates that all the 
participants’ answers were correct on two occasions. 


Figure 3. Changes in the amount of knowledge of the untaught affixes 
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When comparing Weeks 0 and 12, the amount of affix knowledge increased over 
time for ‘-(e)n’ and ‘-ant’. The results suggest that AM10 invoked system learning 
of affixes as opposed to item learning. That is, AM10 contributed to learning of 
the affix system itself, not only the individual affix taught. However, these results 
should be interpreted cautiously, as the participants could have chosen the untaught 
affixes simply because they could tell that the other choices, all of which were the 
learned affixes, were not good matches. 
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Regarding ‘fore-’ and ‘-ity’, the overall increase was limited. Interestingly, there 
was a decrease in knowledge of these affixes at Week 6, which suggests negative 
interference caused by AM10. The current study lacks evidence regarding the cause 
due to the limited number of untaught affixes. At this point, the effect of AM10 on 
untaught affixes remains mixed. Future studies should include other prefixes and 
suffixes that AM10 does not include. 


4. Conclusions 


This study investigated the effectiveness of AM10, a collection of materials which 
teach English affixes systematically. After ten weeks of self-study, the learners 
showed an increase in their affix knowledge. Further, the use of AM10 possibly 
contributed to the system learning of affixes. More studies are needed to generalize 
the findings of this study. However, AM10 showed potential as a teaching/learning 
system of affixes. 
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Learning episodes in an intercultural virtual exchange: 
the case of social high-immersion virtual reality 


Kristi Jauregi-Ondarra', Maria Christoforou’, and Dimitrios Boglou* 


Abstract. Computer-mediated communication tools facilitate international 
collaboration projects between foreign language learners and peers abroad (O’ Dowd, 
2018). Social Virtual Reality (VR) applications allow for synchronous interactions 
and task-based communication in which learners can experience telepresence and 
immersion and conversate ina foreign language. Based on previous pilot experiences 
(Jauregi-Ondarra, Gruber, & Canto, 2020, 2021), this Virtual Exchange (VE) 
project aims to investigate how the specific affordances of Social High-immersion 
VR (SHiVR) in conjunction with designed tasks influence interaction patterns, and 
learning episodes. The VE took place between two groups of university students 
in the Netherlands (N=15) and Cyprus (N=14) through SHiVR in March 2022. 
The main aims of the tasks were to raise student intercultural awareness, stimulate 
task-based communication processes using English as a lingua franca and digital 
pedagogical competences of language education students. Different sources of data 
were gathered and analysed. In this paper, we describe and present the pedagogical 


experience and the initial results. 
Keywords: social high-immersion virtual reality, virtual exchange, language related 
episodes, culture related episodes. 

1. Introduction 

In this study we present the pedagogical experience of university Dutch and 


Cypriot students from a VE project conducted in the immersive environment, 
AltspaceVR, a social application that offers collaboration and communication 
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opportunities to participants who are embodied as customised avatars. In contrast 
to other media, VR affordances enable users to have a multisensory experience 
and simulate situated learning opportunities (Christoforou, Xerou, & Papadima- 
Sophocleous, 2019) that could transform VE projects. Through the use of 
VR-mediated tasks, we tried to explore how the whole experience influences 
interactions and learning episodes using English as a lingua franca. 


2. Method 


The project was embedded in the students’ courses, Language Education and ICT 
(Dutch students), and English for Chemical Engineering (Cypriot students). Prior 
to the collaboration, the Dutch students watched video tutorials and participated 
in preparatory immersive sessions on how to operate the Oculus headsets in VR, 
while the Cypriot ones familiarised themselves with the Oculus Quest 2 headsets 
in a Cyprus University of Technology (CUT) lab. They performed three tasks in 
dyads (getting to know each other, student life, and a task designed by the Dutch 
students). They carried out their sessions at times of their own convenience in 
AltspaceVR. 


Several sources of data have been gathered in order to investigate which learning 
opportunities emerge during the sessions: pre- and post-surveys (including a five 
point Likert scale and open questions); reflection diaries, which the students filled 
in after each task to describe their intercultural experience with the VR app, as 
well as the communication and collaboration with the partner; and interaction 
recordings and focus group interviews with the instructor conducted at the end of 
the project. 


3. Results and discussion 


Because of space constraints, in this section we present first the results of students’ 
general immersive experience based on the final surveys. We then move to analyse 
the interaction affordances of the VR app and show examples of the interaction 
patterns of one dyad (NL1-Cy1)* performing Task 2 (student life). 


4. NL: Dutch student; Cy: Cypriot student 
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3.1. Students’ general experience 


Eleven Dutch and ten Cypriot students filled out the final questionnaire. Students 
found it easy to start and use the VR tool and sound, crucial in communication 
processes, was good (Table 1). Although in general, participants were positive 
about the whole VR exchange experience, we notice a clear discrepancy between 
the Dutch and Cypriot students, the latter ones being much more enthusiastic than 
the Dutch. Interestingly, all students valued the international dimension highly. 


Table 1. Main VR exchange results based on the final survey 


Item NL Cy 

M SD M SD 
It was easy to START the tool. 3.5 0.9 4.3 0.8 
It was easy to USE the tool. 3.9 0.9 4.4 0.7 
SOUND was good. 4.2 1.0 3.6 0.8 
I found the VR environment difficult to use. 1.8 0.9 1.9 0.7 
I like to communicate in this VR environment. 3.3 0.9 4.6 5.5 
I like to meet students from other 3.9 deal 4.5 0.5 
countries in this VR environment. 
I enjoyed the VR sessions. 3.7 0.8 4.5 0.5 
This VR environment is nice. 3.6 0.7 4.7 0.5 
I think VR is an effective system for 3.4 1.1 4.4 0.5 
practising a foreign language. 
I liked the tasks I carried out with my partner. 3.9 0.8 4.6 0.7 
The international dimension (the interaction 4.5 0.8 4.5 0.5 
with students from another country) made 
the VR sessions more interesting. 
The VR sessions helped me discover new things 4.0 1.0 4.4 0.7 
about other people’s views, another culture. 


Note. N(NL)=11; N(Cy)=10. Responses in five point Likert scale (1. Strongly disagree — 5. Strongly agree). M: mean values. 
SD: Standard Deviation. 


3.2. Learning affordances of VR 


During the AltspaceVR interactions (Figure 1), the most prevalent affordance 
was the virtual communication ability, depicted through the participants’ chosen 
avatars and their abilities to gesture during their conversation and point to 
various directions or other virtual elements in the virtual environment. A second 
affordance was the full immersion of the participants during their interaction 
in their virtual world as both saw their avatars as an extension of their bodies 
from the real into the virtual world. All these affordances were displayed during 
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the participants’ verbal interaction and collaboration to teach each other how to 
interact with VR elements and objects found in the AltspaceVR world, in this 
case how to shoot a basketball and how to light and shoot fireworks. 


Figure 1. Students interacting in AltspaceVR 


3.3. Interaction and learning episodes 


Interaction data was analysed to find out how students engaged in social interaction 
in the VR space and which learning opportunities emerged in the sessions. For this 
task students had to preview a 360° video over the university and/or the city of the 
speech partner. 


¢ Culture related learning episodes 


The interaction is seeded with varied and rich culture related episodes (Zakir, 
Funo, & Telles, 2016). NLI initiated the video discussion asking Cyl about 
his global impression about Utrecht. Cyl referred to Utrecht as an ‘amazing 
city’, with ‘the river’ and where ‘everybody is cycling’. They then moved to 
compare both cities and universities in terms of differences and similarities: they 
referred to the density, the size, and colours of the city and the buildings, the city 
planning, and the drivers’ behaviours. NL1 related the general impression to his 
personal experience when visiting the southern European countries. Later in the 
interaction Cyl expressed his wish to visit Utrecht and the Netherlands and NL1, 
Cyprus. Both suggested activities they could undertake during their visit. They 
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then moved to discuss the studies and students’ lives. In the following excerpt, 
we can see how they exchange information about living alone, living with the 
family at home, or in a student house during their study. It is interesting to see 
how they personalise the experience, trying to create bridges, and reach common 
ground. 


Excerpt 1: Culture-related episodes 


NLI1: Maybe, would you ever like to go in a real student house with a lot of other students or would it be 
too busy? 

Cy1: abbh (.) I don't know, I don’t think I would like to go to a student house, like I am I am pretty introvert 
as a person, so I like to have my quiet 

NLI: yep, I really get it too (.). II live still at home with my parents and I think it is just fine for me. I don’t 
want to go to a student house. A friend of mine asked me to join his house, but there are 16 students in one 
house, so I don’t think it would be very good for me= 

CY 1:=oohh, yeaa. That sounds horrible honestly. I wouldn't like to go= 

NLI: =no, me neither. 


e Language related episodes 


The students’ utterances in English in the language episodes were mostly lexis and 
mechanics-oriented. In Task 2, the students talk about languages. 


Excerpt 2: Language-related episodes 


NLI: ...but with English I’m more comfortable so I’m glad we’re doing this in English. 

Cyl: Yeah, it’s definitely a lot better doing that in English. 

NLI: We can also try it in Greek, I know a few words in Greek... 

Cy1: (chuckles) 

NLI: Not too many to have a real conversation, but I can say kalimera (kaAnuépa =good morning in Greek), 
that’s something, right? 

Cyl: Yeah, it is something. 


It is interesting to see how the Dutch student’s metatalk included a Greek word 
which serves as an instance of bridging language and culture. Within the context of 
organising a party and the snacks they usually eat, Cy! says: 


Excerpt 3: Language-related episodes 


Cyl : Another is Jountza ham. 

NLI : Ok, what is that? 

Cyl : Lou-ntza. It’s hard to pronounce. 
NL1 : Lou-ntza? 

Cyl : Yeah, you got that right! 


This is a mechanics-oriented short exchange, depicted through the students’ wish 
to pronounce the local word right, followed by a rewarding reassurance from Cy1 
at the end. 
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4. Conclusions 


In general, students enjoyed the intercultural learning experience, although Cypriot 
students were more positive about the immersive experience than their Dutch peers. 
The analysed interaction shows that a task seeded with cultural elements enhances 
rich intercultural negotiations in the immersive environment. Interestingly, instead 
of providing just global information on cultural issues and falling into stereotypes, 
students, embodied as avatars, succeed in personalising their cultural experience, 
voice their cultural identity, learn from one another, and reach common ground. 
These are preliminary results that will need to be substantiated with additional data 
analysis. 
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Evaluating automatic speech recognition for L2 
pronunciation feedback: a focus on Google Translate 


Paul John', Walcir Cardoso’, and Carol Johnson? 


Abstract. This study examines the L2 pronunciation feedback provided by the 
Automatic Speech Recognition (ASR) functionality in Google Translate (GT). 
We focus on three Quebec Francophone (QF) errors in English: th-substitution, 
h-deletion, and h-epenthesis. Four hundred and eighty male and female QF recordings 
of sentences with correctly and incorrectly pronounced final items (e.g. J dont 
know who to thank versus tank) were played into GT. Errors were equally divided 
between mispronunciations leading to real word (thank — tank) and nonword output 
(thief — tief). As anticipated, we found greater transcription accuracy for correct 
pronunciations and, among incorrect pronunciations, for real words versus nonwords. 
Overall, our findings suggest ASR can be highly effective for pronunciation feedback. 
We also examined transcriptions for gender bias, since ASR systems are often trained 
on corpora with more male voices, but our concerns proved unfounded: surprisingly, 


higher transcription accuracy was found for female recordings. 


Keywords: automatic speech recognition, Google Translate, L2 pronunciation, 
corrective feedback, gender bias. 


1. Introduction 


ASR technology constitutes a promising means for second language (L2) learners 
to access feedback on pronunciation errors. For example, QFs typically struggle 
with English /0/ and /h/, tending to substitute /t/ for /@/ (thank — tank), and to 
delete or epenthesize /h/ (heat > _eat / air — hair respectively) (Brannen, 2011; 
John & Cardoso, 2009). If QFs use ASR to transcribe their output for targets 
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such as thank, heat, and air (e.g. in a reading-aloud task), they may find that 
the transcription reflects the incorrect pronunciations tank, eat, and hair. The 
transcription thus provides the invaluable feedback that learners have produced 
instances of th-substitution, h-deletion, and h-epenthesis, and learners can revise 
their pronunciation until the transcription matches the target. 


Corrective feedback is an effective means of promoting L2 pronunciation accuracy 
(Lyster, Saito, & Sato, 2013). Immediate feedback is, however, hard to provide, 
and delayed feedback (e.g. on recordings) is time-consuming for teachers to 
formulate, so learners may not receive much feedback. This is where ASR can 
fill the gap, generating feedback that learners access ‘anytime-anywhere’ to 
engage in autonomous learning (van Lieshout & Cardoso, 2022). Nonetheless, 
questions remain regarding the adequacy of ASR-generated feedback, as in the 
widely available tool GT. To what extent do GT transcriptions capture correct and 
incorrect pronunciation? 


Our study investigates transcription accuracy for items appearing in sentence 
contexts rather than in isolation (e.g. in wordlists). In this case, GT can identify the 
target item using not only phonetic but also contextual (syntactic/collocational/ 
semantic) cues (Ashwell & Elam, 2017). Under these conditions, we expected 
higher transcription accuracy for correctly than incorrectly pronounced target 
items. Given correct pronunciation (J dont know who to thank), phonetic and 
contextual cues converge on the target item. Given incorrect pronunciation 
(I dont know who to tank), phonetic and contextual cues conflict, and GT 
may transcribe contextually motivated thank rather than phonetically accurate 
tank, thereby failing to flag the pronunciation error. The adequacy of ASR- 
based feedback depends on transcriptions both confirming correct and flagging 
incorrect pronunciations. 


With incorrect pronunciations, we also anticipated greater accuracy for real 
words (thank — tank; heat > _eat; air — hair) versus nonword output (thief > 
tief; head — _ead; ice — hice). Nonwords being absent from the GT lexicon, 
the technology should fail to flag such errors, often supplying the contextually 
appropriate item. Finally, we investigated ASR for gender bias: since ASR is often 
trained on corpora with more male voices (Tatman, 2017), we anticipated a male 
transcription advantage. 


The following summarizes our predictions regarding transcription accuracy (with 
‘>? indicating ‘greater than’): correct > incorrect pronunciations; real word > 
nonword output; male > female speech. 
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2. Method 


One hundred and twenty sentences were recorded by four male (M) and four 
female (F) QF adults with correct and incorrect pronunciation of final items starting 
with /0/, /h/, or a vowel. Among incorrect pronunciations, 60 led to real word and 
60 to nonword output. Four hundred and eighty recordings (four versions of each 
sentence: 1M/IF recording with correct/incorrect pronunciation) were played into 
GT and coded for final-item transcription accuracy. 


Among inaccurate transcriptions for correctly/incorrectly pronounced items, we 
also determined rates of ‘false alarms’ and ‘false negatives’. A false alarm involves, 
for example, correctly realized thank being transcribed as tank, misleadingly 
suggesting the learner has substituted /t/ for /0/. A false negative involves thank 
being transcribed as thank, despite being incorrectly realized as tank, misleadingly 
indicating target-like pronunciation. 


3. Results and discussion 


Table | presents accuracy rates for transcriptions of correctly pronounced sentence- 
final items. The overall accuracy rate (88.33%) for correct pronunciations is 
reassuring: in most cases, GT confirmed the target-like pronunciation. 


Furthermore, among inaccurate transcriptions (11.66%), we can report that fully 
half (5.83%) constitute ‘near-accurate’ transcriptions. That is, the mistranscription 
nonetheless started with the problematic target sound (e.g. output thrifty was 
transcribed as thirsty), from which learners can correctly conclude that they 
successfully realized the target sound (/@/ in this example). In addition, no false 
alarms occurred among the mistranscriptions: correctly pronounced thank-heat-air 
were never transcribed as tank-eat-hair. Again, the absence of such misleading 
feedback is encouraging. 


Table 1. Transcription accuracy: correct pronunciations (%) 


Target items M F M+F 
th-initial 72.50 85.00 78.75 
h-initial 90.00 97.50 93.75 
V-initial 87.50 97.50 92.50 
Mean 83.33 93.33 88.33 
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Table 2 presents accuracy rates for transcriptions of incorrectly pronounced 


sentence-final items leading to real word (a) and nonword output (b). 


Table 2. Transcription accuracy: incorrect pronunciations (%) 


a. Real word output (thank — tank) 

Target items M F M+F 
th-initial 30.00 40.00 35.00 
h-initial 35.00 65.00 50.00 
V-initial 30.00 85.00 57.50 
Mean 31.66 63.33 47.50 
b. Nonword output (thief — tief) 

Target items M F M+F 
th-initial 0.00 0.00 0.00 
h-initial 5.00 15.00 10.00 
V-initial 10.00 20.00 15.00 
Mean 5.00 11.66 8.33 


As expected, the overall mean for incorrect pronunciations resulting in real words 
(47.5%) is considerably lower than for correct pronunciations (88.33%). The 
mean for nonword output (8.33%) is lower still, so GT is virtually incapable of 
providing feedback on nonword mispronunciations. Exceptions were mainly due 
to the technology identifying proper nouns corresponding to phonetic input (e.g. 
oil — hoil was transcribed as Hoyle, a place name). This is important information 
for teachers who wish to design ASR-based pronunciation activities: these should 
target items that, if mispronounced, lead to real word output. Since GT flags almost 
half the real word errors here, it constitutes an effective tool for providing QF ESL 
learners with pronunciation feedback. Moreover, some mistranscriptions (14.5% in 
all) were ‘near-accurate’: for example, output ‘eft for theft was transcribed as fest, 
which captures the mispronunciation. 


Nonetheless, GT generated numerous false negatives: 36.66% (real word output) 
and 65% (nonword output). Consequently, to verify pronunciation ability, learners 
should test themselves on multiple tokens, practicing until transcriptions are 
consistently accurate. Alternatively, learners can produce items in isolation, thus 
eliminating the possibility that GT bases its transcription on contextual cues. 


Interestingly, contra the expected pattern for gender bias, transcription accuracy for 
F recordings is higher than for M recordings across the board, whether for correct 
(Table 1) or incorrect pronunciations (Table 2). Conceivably, the F recordings 
contained the more careful pronunciation that typifies female L2 speech (Moyer, 
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2016). This hypothetically clearer articulation may facilitate automatic recognition 
of individual items and override any inherent gender bias in the technology. 


4. Conclusions 


Our findings indicate that ASR is a highly promising tool for much-needed L2 
pronunciation feedback. GT showed high transcription accuracy for correct 
pronunciations and no false alarms. Moreover, the consistently higher transcription 
accuracy for female voices suggests that concerns about ASR gender bias are 
unfounded. Although ASR struggles with nonword errors (8.33% transcription 
accuracy), almost half of real word errors in a sentence context were flagged, and 
we would expect even higher transcription accuracy for items spoken in isolation. 
Indeed, the next stage of research will target wordlists, such that transcriptions are 
based on phonetic input alone, without contextual cues. Future research could also 
go beyond consonants to include L2 vowel and lexical stress errors. In sum, while 
the technology has certain limitations for teachers to highlight and explain, L2 
learners can use ASR to access invaluable feedback on pronunciation. 
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Abstract. This study examined the use of a popular Automatic Speech Recognition 
(ASR), Google Voice Typing (GVT), to automatically assess English as second 
language pronunciation. It aimed to answer the following question: What is the 
relationship between GVT-rated scores and human-rated scores? To answer this 
question, we compared audio recordings of 56 oral placement tests, rated by both 
human raters and GVT. Our results indicate that GVT scores strongly correlated 
with human-rater scores, indicating that this non-customizable ASR technology 
could be leveraged to increase the test usefulness of language placement tests. 


Keywords: automatic speech recognition, automatic assessment, L2 pronunciation, 
Google voice typing. 


1. Introduction 


Language programs and schools rely on in-house placement tests to ensure 
students register in level-appropriate classes. However, these tests require 
extensive financial and human resources (Isaacs, 2018). This is especially true 
when assessing pronunciation, which often involves interviewers and multiple 
raters (Cox & Davis, 2012). However, there are known problems with rater 
reliability. For example, raters may overestimate the comprehensibility of second 
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language (L2) speakers when familiar with their accents (Carey, Mannell, & 
Dunn, 2010). This lack of reliability can lead not only to the incorrect placement 
of students, but also to skepticism of test results (van der Walt, de Wet, & Niesler, 
2008). 


Using ASR to assess pronunciation could mitigate these problems. Large 
testing companies have been using custom-built ASR technology for more than 
two decades (e.g. Pearson’s Versant — Bernstein, Van Moere, & Cheng, 2010). 
These organizations have access to financial and human resources beyond that 
of language institutions, but with advances in technology, it is now feasible for 
smaller organizations to take advantage of ASR to assess pronunciation. Studies 
of customized ASR-based assessment tools developed for language placement 
tests have found that ASR scores are strongly correlated with human-rater scores 
(Cox & Davis, 2012, van der Walt et al., 2008). Nevertheless, customizing ASR 
still requires a substantial budget and specific knowledge for development and 
maintenance (Isaacs, 2018). One way of mitigating this might be the use of free 
non-customizable ASR such as GVT, which has reached a high recognition rate 
of English L2 speech for high proficiency learners (McCrocklin & Edalatishams, 
2020) and, consequently, has the potential to provide language institutions with 
simple low-cost solutions for pronunciation assessment. 


The aim of this study was to determine if the use of GVT to assess pronunciation 
could increase the test usefulness of a university language placement test, based 
upon Bachman and Palmer’s (1996) test usefulness model. This model consists 
of six qualities that determine the usefulness of a test: reliability, construct 
validity, authenticity, interactiveness, impact, and practicality. The relative 
importance of these qualities depends on the context of the test. As such, there 
is no perfect intertwining of the qualities. Instead, test developers must balance 
these qualities and accept that some may be negatively impacted for the sake of 
others, based on the purpose of the test. The research question that guided this 
study was: 


¢ (1) What is the relationship between GVT-rated and human-rated 
pronunciation scores? 


¢ (la) Do relationships vary between GVT-rated and human-rated scores 
across a set of evaluation criteria? 


¢ (1b) Do relationships vary between GVT-rated and human-rated scores 
across participant proficiency levels? 
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2. Method 


Fifty-six undergraduate students of various oral proficiency levels at a university in 
Canada were recorded during the pronunciation portion of their placement tests for 
English second language (ESL) courses (Novice=2, Beginner=6, Intermediate=12, 
Advanced=14, and Fluent=22). Participants read aloud five increasingly difficult 
sentences (randomly chosen from a bank of sentences), which appeared sequentially 
on a screen for a period of 20 seconds each. To obtain the human-rated score, 
a rubric assessing five criteria (comprehensibility; phonemes; connected speech; 
word stress and rhythm; thought groups, sentence stress, and intonation) was 
used by three experienced ESL instructors who came to a consensus about each 
participant’s score for each criterion. The same recordings were then played into 
GVT in Google Docs to obtain the GVT score. The output was analyzed manually 
with a point given for each correctly recognized word. The total number of points 
was divided by the total number of words in the sentences and multiplied by 100. 
Correlations were run to determine if a relationship existed between the human- 
rated and the GVT scores. 


3. Results and discussion 


The results are summarized in Table | below. In answer to the first research 
question regarding the relationship between human-rated and GVT-rated scores, 
a statistically significant strong correlation was found between human-rated and 
GVT scores. Regarding the first sub-question about relationships between GVT 
scores and the rubric criteria (1a), statistically significant strong correlations were 
also found between each criterion and the GVT scores. In regard to the second 
sub-question concerning the relationship between GVT scores and test-taker 
proficiency (1b), a significant strong correlation was found for lower-proficiency 
test-takers (Beginner-Intermediate), but a non-significant weak correlation was 
found for higher-proficiency test-takers (Advanced-Fluent). 


These findings corroborate the existing literature that found ASR scores correlate 
with human-rater scores (Cox & Davies, 2012; van der Walt et al., 2008). This 
study further contributes to the field as our findings seem to indicate that non- 
customizable ASR, such as GVT, is a valid option when automatizing the 
assessment of pronunciation. 


In terms of Bachman and Palmer’s (1996) test usefulness model, the use of GVT 
can improve the overall test usefulness of a pronunciation placement test. Validity, 
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reliability, and practicality are the only three elements of the model affected by 
changing the assessment method. Validity may be somewhat reduced as GVT 
assessed intelligibility (e.g. what it understands) whereas the human raters 
assessed multiple aspects of pronunciation (e.g. comprehensibility and prosody). 
However, it allows for an increase in both reliability (i.e. the consistency of 
machine scoring) and practicality (i.e. reduced human resource costs and time 
to score tests). 


Table 1. Descriptive statistics and correlations: GVT scores 


Variable M SD rho p 95% 
BCa 
Cis 
GVT Score (/100) 73.09 |22.49  - - - 
Human-rater score (/100) 72.00 | 26.95 .78** = <001 | .64,.88 
Comprehensibility (/5) 4.14 1.20 85** =} <.001 | .74,.90 
Phonemes (/5) 3.39 1.47 .78**  <001 | .63,.88 
Connected speech (/5) 3.34 1.51 .72** =| <,001 — .53,.84 
Word stress and Rhythm (/5) 3.63 1.34 71**®  <001 | .52,.84 
Thought groups, sentence 3.50 1.51 79** <001 | .65,.88 


stress, intonation (/5) 
Lower-proficiency — GVT score (/100) 47.74 = 13.78 | - - - 


Lower-proficiency — Human- 40.20 |16.80 |.78** <001 | .56,.89 
rater score (/100) 


Higher-proficiency — GVT score (/100) 87.18 | 10.96 — - - - 


Higher-proficiency — Human- 89.67 9.44 28 .10 -.07,.56 
rater score (/100) 


Note. Confidence intervals based on 1,000 bootstrap samples. 
*y<.001. 


4. Conclusions 


The purpose of this study was to explore the use of GVT to score a pronunciation 
assessment. The findings reported indicate that GVT scores strongly correlated with 
human-rater scores, suggesting that non-customizable ASR could be leveraged to 
increase the usefulness of a placement test. 


Certain limitations should be taken into consideration. With only 56 participants, the 
findings are not generalizable to other populations. Additionally, recordings were 
used rather than live speech, and at times, this may have affected the technology’s 
ability to correctly represent what was being said, negatively impacting the scores 
generated by GVT. 
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This study has shown the potential of using non-customizable ASR technology to 
score pronunciation placement tests. The next step is to fully automate the scoring 
process. This would provide opportunities to further research its use, such as 
determining if GVT has any biases in terms of first language, gender, or age. 


Language programs and schools rely on placement tests to ensure that the learning 
experiences of students are optimal. Automating the placement process for 
pronunciation courses by using free and readily available technology such as GVT 
would allow institutions to offer their students a more streamlined and reliable 
service. Language programs and schools should consider taking advantage of it to 
facilitate the placement of students while at the same time ensuring that students 
continue to receive the learning experiences they want and deserve. 
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Abstract. Many language learners lack the motivation to read complex texts. 
Because high-immersion Virtual Reality (VR) is increasingly reported to be highly 
motivating, the goal of our study was to compare English as a Second Language (ESL) 
learners’ (N=79) motivation while reading a story with subtitles in VR (experimental 
group) versus reading the same story screencast in two-dimensions (2D — control 
group). The Wilcoxon signed-rank test revealed that learners’ motivation in VR was 
significantly higher compared with the control group. Our finding confirmed that 
VR is highly motivational in the context of reading foreign texts and can be useful 
for Computer Assisted Language Learning (CALL) researchers, practitioners, and 
instructional designers. This analysis is a part of a larger study (Kaplan-Rakowski & 
Gruber, 2023) on VR facilitating reading tasks. 


Keywords: motivation, high-immersion virtual reality, reading, subtitles. 


1. Introduction 


High motivation to practise languages is essential for their mastery. For practising 
complex language skills such as reading, high motivation is important as many 
learners today tend to limit their reading input to short text messages or social 
media posts. Therefore, devising approaches that allow students to read larger text 
passages without becoming demotivated have become increasingly important. 
Immersive technologies have come to the rescue because they can be highly 
engaging, immersive, and motivational (Kaplan-Rakowski & Meseberg, 2019). An 
example of such technology is high-immersion VR. 
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To experience VR, users need to wear a VR headset that serves as a viewing 
and interactional tool. Thanks to headtracking technology, users can experience 
360° scenarios and feel that they are ‘there’, that is, in the middle of the scenario. 
Such a state is often referred to as the sense of presence (Slater, 2018) or 
immersion. Both concepts typically distinguish high-immersion VR from other 
immersive technologies because, as of 2022, no other technology offers as high 
levels of immersion as VR. Our definition of VR is “computer-generated 360° 
virtual space that can be perceived as being spatially realistic, due to the high 
immersion afforded by a head-mounted device” (Kaplan-Rakowski & Gruber, 
2019, p. 552). 


The use of Virtual Reality-Assisted Language Learning (VRALL) has been 
on the rise. While VRALL research has been limited and clear benefits of VR 
learning outcomes are still inconclusive (Dhimolea, Kaplan-Rakowski, & Lin, 
2022), language scholars have been exploring VR affordances to practise various 
language aspects including vocabulary learning (Papin & Kaplan-Rakowski, 
2022), communication (Gruber & Kaplan-Rakowski, 2022), and foreign language 
anxiety (Gruber & Kaplan-Rakowski, 2020, 2023; Thrasher, 2021). 


This study focused on reading subtitles which differs from traditional forms of 
reading. Research on reading subtitles in CALL has been extensive (see, e.g. 
Pattemore & Mufioz, 2020; Winke, Gass, & Sydorenko, 2010), but the new form 
of digital reading of subtitles in VR needs to be explored. Our research question 
was: does reading a story in a foreign language in VR enhance motivation 
compared with reading the same story in a video screencast capture on a 2D 
screen? 


2. Method 


The study sample consisted of 79 intermediate ESL students at a German 
university. The volunteers signed a letter of consent and were instructed 
individually on how to use the VR device (Oculus Quest 2). The participants 
were randomly assigned to two groups and were tasked to read an identical 
20-minute interactive story in English in two different formats. The experimental 
group used high-immersion VR, wearing a VR headset, and the control group 
viewed the screenshot video of the same story on a 2D screen (see Figure |). A 
screenshot of a part of the story read by the students is displayed in Figure 2. The 
groups had a comparable distribution of participants’ gender, age, L1, and L2 
(English) proficiency. 
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Figure 1. Participants reading in VR (left) and on a two-dimensional screen 
(right) 


The story consisted of about 1,000 words, which is equivalent to two double- 
spaced pages. Unlike the control group, the experimental group could interact with 
the setting using the haptic system (i.e. virtual hands) and could benefit from the 
360° experience using headtracking. Such actions allowed for higher immersion 
and a sense of agency (Kaplan-Rakowski & Gruber, 2021). 


Figure 2. A screenshot of a part of the story 


It may be too repetitive for some, 
but not for Pedro, the paperboy. 


Note. The use of virtual hands offers the possibility of interaction within the story. Subtitles help follow the story. 


In addition to the demographic questionnaire eliciting information such as 
gender, age, L1, and L2 level, the participants completed a survey with the goal 
of measuring levels of motivation across conditions. This survey was based on 
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the Intrinsic Motivation Inventory and consisted of nine five-point Likert-scale 
items. The items assessed levels of self-perceived enjoyment, effort, value, and 
competence. The choices of agreement ranged from ‘not at all true’, to ‘somewhat 
true’, to ‘very true’. The instrument had excellent internal reliability (a =.92). To 
calculate differences between the levels of motivation between the two groups, we 
used the Wilcoxon signed-rank test. 


3. Results and discussion 


The Wilcoxon signed-rank test revealed that motivation of the VR group was 
statistically significantly higher than the 2D desktop group (Z=6.33, p=<.0001). 
The robustness of this finding is very strong as indicated by a very high effect size 
(Cohen’s d=1.94). The descriptive statistics of the VR group (n=35) were M=4.50, 
SD=.59, Min=2.14, Max=5.00; of the video group (n=44), they were M=3.12, 
SD=.79, Min=1.57, Max=5.00. 


A systematic review of VRALL research (Dhimolea et al., 2022) showed that 
students reveal high levels of motivation while using VR. Our results add to this 
line of research and confirm results of other language scholars (Alfadil, 2020; 
Kaplan-Rakowski & Wojdynski, 2018; Pack, Barrett, Liang, & Monteiro, 2020). 
However, the context of the other studies differed from ours as they focused 
either on vocabulary learning, communicative skills, or writing, without exploring 
reading in VR. 


Compared to the control condition, the experimental condition in VR was more 
interactive and had an increased sense of agency and immersion. The video 
condition was passive as it involved following the story on a 2D screen which had 
been recorded in VR (see video screencast: https://tinyurl.com/mcbtkyn4). Because 
we did not use eye-tracking technology, the exact ratio of time spent on reading 
the text and exploring other parts of the scenery within the VR story is unknown, 
thus, limiting our study. We could treat our experimental and control conditions 
holistically, assuming that their overall affordances impacted the perceived levels 
of motivation. 


4. Conclusions 


This one-time-intervention study confirmed that VR has potential to motivate 
and engage students in the context of reading comprehension tasks. Further 
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studies should measure other factors such as cognitive load, sense of presence, 
and ability to comprehend the text (Kaplan-Rakowski & Gruber, 2023). Because 
reading tasks are often considered difficult and dull, offering language learners 
the possibility of reading text with subtitles within VR is a viable option that 
would expose them to written input and, consequently, foster the expansion 
of vocabulary, grammar structures, and other key factors that boost language 
proficiency. 


References 


Alfadil, M. (2020). Effectiveness of virtual reality game in foreign language vocabulary acquisition. 
Computers & Education, 153, 103893. https://doi.org/10.1016/j.compedu.2020.103893 
Dhimolea, T. K., Kaplan-Rakowski, R., & Lin, L. (2022). A systematic review of research on 
high-immersion virtual reality for language learning. TechTrends, 3. https://doi.org/10.1007/ 

$11528-022-00717-w 

Gruber, A., & Kaplan-Rakowski, R. (2020). User experience of public speaking practice in virtual 
reality. In R. Z. Zheng (Ed.), Cognitive and affective perspectives on immersive technology 
in education (pp. 235-249). IGI Global. https://doi.org/10.4018/978-1-7998-3250-8.ch012 

Gruber, A., & Kaplan-Rakowski, R. (2022). Verbal and nonverbal communication in high- 
immersion virtual reality for language learners. In B. Arnbjérnsdottir, B. Bédi, L. Bradley, 
K. Fridriksdottir, H. Gardarsdottir, S. Thouésny, & M. J. Whelpton (Eds), /ntelligent CALL, 
granular systems, and learner data: short papers from EUROCALL 2022 (pp. 129-134). 
Research-publishing.net. https://doi.org/10.14705/rpnet.2022.61.1447 

Gruber, A., & Kaplan-Rakowski, R. (2023). The impact of high-immersion virtual reality 
on foreign language anxiety when speaking in public. SSRN. https://doi.org/10.2139/ 
ssm.3882215 

Kaplan-Rakowski, R., & Gruber, A. (2019). Low-immersion versus high-immersion virtual 
reality: definitions, classification, and examples with a foreign language focus. Proceedings 
of the Innovation in Language Learning International Conference 2019 (pp. 552-555). 
Pixel. 

Kaplan-Rakowski, R., & Gruber, A. (2021). One-on-one foreign language speaking practice in 
high-immersion virtual reality. In Y. J. Lan & S. Grant (Eds), Contextual language learning— 
real language learning on the continuum from virtuality to reality (pp. 187-202). Springer. 
https://doi.org/10.1007/978-98 1-16-3416-1 

Kaplan-Rakowski, R., & Gruber, A. (2023). An experimental study on reading in high-immersion 
virtual reality. SSRN. https://ssrn.com/abstract=4262 124 

Kaplan-Rakowski, R., & Meseberg, K. (2019). Immersive media and their future. In R. M. 
Branch et al. (Eds), Educational media and technology yearbook (vol. 42, pp. 143-153). 
Springer. https://doi.org/10.1007/978-3-030-27986-8 13 


212 


Motivation and reading in high-immersion virtual reality 


Kaplan-Rakowski, R., & Wojdynski, T. (2018). Students’ attitudes toward high-immersion 
virtual reality assisted language learning. In P. Taalas, J. Jalkanen, L. Bradley, & S. Thouésny 
(Eds), Future-proof CALL: language learning as exploration and encounters — short papers 
from EUROCALL 2018 (pp. 124-129). Research-publishing.net. https://doi.org/10.14705/ 
rpnet.2018.26.824 

Pack, A., Barrett, A., Liang, H., & Monteiro, D. V. (2020). University EAP students’ perceptions 
of using a prototype virtual reality learning environment to learn writing structure. 
International Journal of Computer-Assisted Language Learning and Teaching, 10(1), 27-46. 
https://doi.org/10.4018/IJCALLT.2020010103 

Papin, K., & Kaplan-Rakowski, R. (2022). A study on vocabulary learning using immersive 360° 
pictures. Computer Assisted Language Learning, 35. https://doi.org/10.1080/09588221.202 
2.2068613 

Pattemore, A., & Mufioz, C. (2020). Learning L2 constructions from captioned audio-visual 
exposure: the effect of learner-related factors. System, 93, 102303. https://doi.org/10.1016/j. 
system.2020.102303 

Slater, M. (2018). Immersion and the illusion of presence in virtual reality. British Journal of 
Psychology, 109(3), 431-433. https://doi.org/10.1111/bjop.12305 

Thrasher, T. (2021). The impact of virtual reality on L2 French learners’ language anxiety and 
oral comprehensibility: an exploratory study. CALICO Journal, 39(2), 219-238. https://doi. 
org/10.1558/cj.42198 

Winke, P., Gass, S., & Sydorenko, T. (2010). The effects of captioning videos used for foreign 
language listening activities. Language Learning & Technology, 14(1), 65-86. 


213 


EU 


Students’ perceptions of digital 
storytelling in online EFL classes 
with Zoom at a Japanese university 


Naoko Kasami! 


Abstract. This study reports on the results of students’ perceptions of Digital 
Storytelling (DS) in online classes enhanced with synchronous Zoom meetings. Two 
main problems were identified with remote English as a Foreign Language (EFL) 
courses conducted primarily as asynchronous classes in spring 2020. Firstly, seven 
of the 64 participating students (10.9%) found the DS assignment very difficult. 
Secondly, there should be more opportunities for students to learn from others. 
In order to solve these problems, the courses in spring 2021 were enhanced with 
synchronous (90-minute) Zoom meetings held approximately every two weeks to 
increase interaction and support in the classes. The analysis of the post-questionnaires 
showed that there was a slight improvement, with only five out of 77 students (6.5%) 
reporting that they found the DS assignment very difficult; most students (92.2%) 
rated the difficulty level of the assignment as appropriate. Regarding learning from 
others, there were more positive responses from students in 2021. 


Keywords: digital storytelling, online learning, Zoom, EFL. 


1. Introduction 


During the COVID-19 pandemic, there was a rapid transition from classroom 
learning to online learning. In the field of Computer Assisted Language Learning 
(CALL), there have been positive findings in regard to the implementation of 
videos in language education (Yu & Zadorozhnyy, 2022), and synchronous 
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learning with video communication apps such as Zoom and Google Meet have 
become widely used. Numerous studies have demonstrated the advantages of 
using ‘breakout rooms’ in these apps (Kohnke & Moorhouse, 2020; Siang & 
Mohamad, 2022). 


At EUROCALL 2021, the author reported students’ perceptions of DS in 
primarily asynchronous EFL classes at a Japanese university (Kasami, 2021). 
The study reported on findings from asynchronous classes held during spring 
2020 due to COVID-19, using pre-recorded materials, which were supported 
by complementary short (30-minute) synchronous online meetings every few 
weeks. There were two main problems with the use of primarily asynchronous 
classes. Firstly, as seven of the 64 participating students (10.9%) found the DS 
assignment very difficult, there was a need for more appropriate language and 
technical support. Secondly, there was a need to provide more opportunities for 
students to have collaborative and interactive learning. 


In order to solve these problems, the courses were redesigned. The courses in 
spring 2021 were enhanced with synchronous (90-minute) Zoom meetings held 
approximately every two weeks. During these meetings, students were encouraged 
to communicate in groups through the use of ‘breakout rooms’. The rooms 
encouraged students to speak and write in English with others and to provide peer- 
reviews of their assignments. They were also expected to ask questions during the 
meetings or at the end of the meetings. A few optional meetings were held only for 
those who wished to ask additional questions or who requested individual support. 
In addition, answers to frequently asked questions related to DS were revised and 
shared to support students. 


Thus, the purpose of the present study was to examine students’ perceptions of the 
DS in online classes enhanced with synchronous Zoom meetings in EFL courses 
which ran from April to July 2021. 

This study aimed to answer three research questions: 


RQI1. How difficult was the DS assignment? 


RQ2. What were students’ perceptions of learning from others and the 
frequency of the meetings? 


RQ3. What were the pros and cons of the DS assignment from the student’s 
perspective? 
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2. Method 


2.1. Participants 


Most students were basic or intermediate level English learners (from Al to B1 on 
the Common European Framework of Reference for Languages proficiency levels) 
majoring in Information and Communications Technology (ICT) related fields. In 
spring 2021, 90 students enrolled on three elective EFL courses for second to fourth 
year students, and 77 students agreed to participate in this research by answering 
a post-questionnaire survey. The study goal of the course was to acquire skills 
and knowledge to present ideas and messages effectively using ICT and English. 
DS was the main assignment of the course. The amount and conditions of the DS 
assignments were identical to the previous year. 


2.2. Data collection 


In the course, three questionnaires were employed during the term, as had been 
used in the past courses (Kasami, 2021). The post-questionnaires consisted of four 
sections (general impression of the remote course, students’ perceptions of DS, 
self-evaluation of DS, and learning motivation) in the same format as previous 
questionnaires. Like the previous study (Kasami, 2021), questions on students’ 
perceptions of DS were generated in line with studies by Jamieson, Chapelle, and 
Preiss (2005, pp.131-133). Only a few questions on the general impression of 
the remote course and students’ perceptions of DS in the post-questionnaire were 
focused upon in this study. The post-questionnaires were conducted at the end of 
the course and were created using Google Forms with a combination of closed and 
open-ended questions. 


3. Results and discussion 


3.1. _ Difficulty and ease of the assignment 


The aspects of ‘learner fit’ and ‘practicality’ analyzed in the previous study (Kasami, 
2021) were also focused upon in this study. 


In spring 2020, students were asked about the difficulty of the DS assignment 
through the question ‘how difficult was the DS assignment for you?’ (Figure 1). 
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Though most of the students (87.5%) reported that the DS assignment was set at 
a good level of difficulty, 10.9% of students felt it was very difficult. In spring 
2021, fewer students (6.5%) answered that it was very difficult, and more of the 
students (92.2%) reported that the DS assignment was set at an appropriate level 
of difficulty. 


Figure 1. Learner fit 


Learner fit 
How difficult was the DS assignment for you? 


2020 (n=64) 2021 (n=77) 
vey ear Very difficult vey Very difficult 
, is : 5, 

1.6% _, (10.9% 13%. 6.5% 


\ 


Good 


Good 


(adequate) (adequate) 


While in spring 2020, 78.1% of students answered positively to the question “were 
the directions easy to understand in the DS assignment’, in spring 2021, 87% of 
students answered positively (Figure 2). 


Figure 2. Practicality 


Practicality 
Were the directions easy to understand in the DS assignment? 


2020 (n=64) 2021 (n=77) 


No, 0, 
0.0% 


No, 2, 
2.6% 


Some, 8, 
10.4% 


3.2. Learning from others in Zoom meetings 


Regarding collaborative learning, for the statement ‘I was able to learn from others’, 
the students were asked to indicate their degree of agreement on a five-point Likert 
scale (1-Strongly Negative, 2-Negative, 3-Neutral, 4-Positive, 5-Strongly Positive). 
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There were more positive responses from students in 2021. Positive responses 
(4-Positive, 5-Strongly Positive) were received from 53.2% of the students in spring 
2020, but this improved to 70.1% in spring 2021 (Figure 3). 


Figure 3. Learning from others 


Learning from others 
1 was able to learn from others. 


2020(n=64) 2021(n=77) 
1-Strongly 


Negative 1-Strongly 


4.7% Negative __ 
2-Negative __ 2.6% 


47% - 2-Negative_——_y 
5-Strongly 9.1% 
Positive 
18.8% 


4-Positive 
34.4% 


4-Positive 
54.5% 


As for the frequency of meetings, the number of students who answered 
‘appropriate’ decreased from 93.8% in 2020 to 75.3% in 2021. At the same time, 
the number of students who answered ‘slightly too many/frequent’ or ‘too many/ 
frequent’ increased from 4.7% in 2020 to 23.4% in 2021. In both years, only one 
student answered ‘slightly less frequent’, and nobody answered ‘too infrequent’ 
(Figure 4). 


Figure 4. Frequency of meetings 


Frequency of online meetings 


How was frequency of online Zoom meetings? 


2020(n=64) 2021(n=77) 
Slightly less a Pe : ; Too 
frequent memyede Slightly less —_ many/Frequent 
1.6% 4.7% frequent 3.9% 


1.3% 


Slightly too 
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19.5% 
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3.3. | What were the pros and cons of the DS assignment 


Two open-ended questions were presented, and answers were solicited from 
students. For the question ‘what points were preferable in the DS assignment?’, 
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there were 26 responses as follows. All the responses were separated into six 
categories according to common words: (1) ‘Learning from others’ (n=10), (2) 
‘Express/Think’ (n=4), (3) ‘ICT skills’ (n=4), (4) ‘Teacher feedback’ (n=3), (5) 
‘Interesting/Fun’ (n=3), and (6) ‘Sense of accomplishment’ (n=2). Figure 5 shows 
examples of students’ comments. 


Figure 5. Students’ comments 


Pros 


(What points were preferable in the DS assignment?) 


SLearn from others 
“| am glad that | was able to positively face English learning in order 
to understand the contents by watching the works of other 
students.” 

Express / Think 
“It was my first time to make my own work in English, so | had 
various experiences from how to write sentences to how to make 
presentations.” 

MTeacher Feedback 
“Teacher feedback made me motivated for learning.” 


Cons | 


(What points were needed to be improved in the DS assignment?) 


@Time for peer-review 


“| wish | had more time to see other people's work.” 


SDifficulty in finding instructional information 


“If it was clearly written how long a digital story should be, it might 
be better to know the goal. (I'm sorry if it was written.)” 


For the question ‘what points needed to be improved in the DS assignment’, there 
were seven responses, and the common words were (1) ‘Time for peer-review’ 
(n=2) and (2) ‘Difficulty in finding instructional information’ (n=2). 


These comments showed that more Zoom meetings enabled students to have 
more learning opportunities with classmates which consequently made it easier to 
complete the DS assignment. The frequency and length of meetings and assignments 
were generally regarded as appropriate, but there were also some students who 
preferred fewer meetings as they seemed to proceed with their assignments on 
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their own. On the other hand, a few students preferred having more interactions 
with others and additional support to understand what to do. Based on various 
comments, it would be important for teachers to engage students’ personal voices 
and exploit digital technology appropriately in language education as outlined by 
Ushioda (2011). 


4. Conclusions 


The DS assignments were conducted more effectively with other students in 
online classes when enhanced with synchronous Zoom meetings compared 
with primarily asynchronous classes. Yu and Zadorozhnyy (2022) observed 
that creating video was perceived positively though some students experienced 
difficulties due to lacking some skills and knowledge. In accordance with this, 
most students perceived the difficulty level and learning from others positively 
while there were also some students who felt the assignment was difficult and 
advanced students who did not need some meetings. These various responses 
suggest that future courses and study may benefit from different lesson styles 
(such as face-to-face, hybrid, synchronous, and asynchronous) which students 
can select according to their preferences and conditions. 
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Teachers’ physiological signals to improve 
teacher-student relationships 


Olivia Kennedy’, Sandra Healy”, Chie Fukada’, 
and Noriaki Kuwahara‘ 


Abstract. In this study we explore how the physical movements teachers use can 
lead to improved interactions between students in a university language course. The 
study used video to capture and analyse an intervention focusing on the effects of 
teacher nodding. Results showed that positive measurable differences were found in 
students’ physical postural responses in relation to the frequency with which their 
teacher nodded. The next stage of this preliminary research project will make use 
of the data gathered on micro-level interactions to develop technological support 
mechanisms to be used in the classroom to support teacher-student interactions. 


Keywords: physiological signals, teacher-student relationship, body posture, 


student interaction, micro-level interactions. 


1. Introduction 


Positive teacher-student relationships are known to support student learning (Allen 
et al., 2021), and there are many standard strategies that teachers employ to build 
positive relationships within their classrooms. It is important, however, to further 
foster classroom relationships for the benefit of both teachers and students and 
to harness technology to do so. Micro-level interactions affect how classroom 
relationships are perceived (Wubbels, 2018) and can be captured, analysed, and 
then used to improve interactions via technological intervention. In this study 
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we explore how the physical movements used in teaching can lead to improved 
interactions between students. We hope to build a base of knowledge to allow for 
technological classroom support mechanisms to be effectively developed. The 
study utilises embodied cognition, the ways that “body movements both express 
and influence how people feel and think” (Chandler & Schwarz, 2009, p. 123), and 
specifically focuses on the effects of teacher nodding. 


A wide variety of experiments exploring embodiment show that our “thoughts, 
feelings and behaviours are grounded in bodily interaction with the environment” 
(Meier, Schnall, Schwarz, & Bargh, 2012, p. 705). Some of these experiments 
show, for example, that facial expressions (Buck, 1980), arm movements (Forster 
& Strack, 1997), or how often we nod (Brifiol & Petty, 2003; Wells & Petty, 1980) 
can affect our mental processes including the ways that we perceive ourselves and/ 
or others (Mussweiler, 2006). Nodding is almost universally understood to be a 
positive signal (Helweg-Larsen, Cunningham, Carrico, & Pergram, 2004) that 
previous studies suggest correlates with likeability (Godfrey, Jones, & Lord, 1986). 
Some studies suggest that the chameleon effect, seen when a person unconsciously 
copies the physical behaviours of the people that they are with, eases interactions 
between people and increases how positive they feel towards one another (Chartrand 
& Bargh, 1999). In addition, as face coverings are presently omnipresent to contain 
the spread of COVID-19 and may remain so indefinitely, it has become important 
to explore these physical responses. The work of, for example, de Gelder (2009), 
which finds that physical responses such as torso and head angle show more 
consistent and reliable information than facial responses, is particularly reassuring 
in this context. This preliminary research article investigates whether increased 
nodding by the teacher has an impact on student-to-student interaction and posits 
that the enthusiasm displayed by students as teacher nodding increases may be due 
to more positive perceptions of the teacher and, by extension, student relationships 
with them. 


2. Method 


Classroom observations were undertaken in a compulsory first-year English as 
a foreign language class in a Japanese university with an experienced female 
instructor in Week 13 ofa 15-week course. The two female and 14 male participants 
are between 18 and 20 years old. All participants were informed that their class 
would be observed and digitally recorded one week prior and gave informed 
consent according to institutional policies. In line with institutional COVID-19 
practice, all participants wore face coverings. 
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Three video cameras were placed in the classroom; one wide-lens camera 
trained on the teacher, and the others placed on either side of the rear. These also 
recorded the ambient classroom sound on internal microphones. During the first 
45 minutes of the 90-minute observation, the teacher was asked to teach as usual. 
During the second 45 minutes, the teacher was prompted intermittently with a 
visual cue from the rear of the classroom to increase nodding during her next 
interaction. Data was coded using ELAN (2022), with attention paid to teacher 
nodding during teacher-student interactions and to the angle and direction of 
student torsos and gaze during student-student interactions. The students first 
read a short passage individually then completed several short discussion 
activities in pairs. Between discussions, the teacher called for attention and gave 
general feedback before setting the next task. 


3. Results and discussion 


Two three-minute segments of the 90-minute lesson were chosen for comparison. 
In both segments, the teacher introduced a discussion topic, then asked the class 
to talk with a partner. The segments selected were the fourth and fifth discussions 
for the lesson. In the first, the teacher taught as she normally would, nodding 
in encouragement to students who asked questions and interacted with her, and 
to signal the end of her conversational turn. In the second, following the cue to 
increase nodding, the teacher nodded more often, and the vertical range of head 
motion was much greater. In addition to the functions mentioned above, she also 
nodded decisively when choosing an example to illustrate her point, and when 
setting the time limit for the class to undertake the activity. 


The way that student participants undertook pair-work activities in these two 
segments was markedly different. The first of these is illustrated by Figure |. This 
video still was taken 14s03ms milliseconds after the students were asked to begin 
the first activity. Nine participants are visible, eight students and the teacher. Pair 1 
is seated closest to the camera, and Pair 4 furthest away. 


In the first segment, the time between the teacher asking the class to begin the 
activity and the first utterances from the student participants ranged between 
1s37ms and 6s45ms. It took between 3s09ms and 10s22ms for the four pairs 
of students to reach their closest physical positions to one another. Postural 
mirroring was achieved by three of the four pairs within 1m46s68ms. In Figure |, 
Pair 2 can be seen to have the same torso direction and angle, and a shared focal 
point. While Pair 1, closest to the camera, began talking first of the four pairs 
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visible, they did not make eye contact, share a joint focal point, or mirror one 
another’s torso angle during this segment. 


Figure 1. Screenshot of Activity 4 (14s03ms milliseconds elapsed) 


« anak 


In the second segment, however, all parameters occur measurably more quickly. 
As can be seen in Figure 2, taken 5s00ms from the start of the activity, Pair 1 have 
turned their torsos parallel to one another, a mirroring action that was not fully 
achieved by any of the pairs in the previous segment. They also share eye contact. 
Pair 2 are also sharing a focal point, leaning towards one another, and mirroring 
posture with one dropped shoulder. 


Figure 2. Screenshot of Activity 5 (5s0O0ms milliseconds elapsed) 


4. Conclusions 


While only three parameters are analysed here in a small number of participants, the 
measurable differences in their postural responses show clearly that the frequency 
with which their teacher nods has a positive effect on the learning environment. The 
next stage of this preliminary research project will make use of the data gathered 
on micro-level interactions to develop technological support mechanisms to be 
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used in the classroom to support teacher-student interactions. Under consideration 
is advice to prompt the teacher to nod more frequently, and thereby ensure the 
continued impact of this low-investment, high-return intervention. 
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Learning L2 pronunciation with Google Translate 
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Abstract. This article, based on Khademi’s (2021) Master’s thesis, examines the 
use of Google Translate (GT) and its speech capabilities, Text-to-Speech Synthesis 
(TTS) and Automatic Speech Recognition (ASR), in helping L2 learners acquire the 
pronunciation of English past -ed allomorphy (/t/, /d/, /id/) in a semi-autonomous 
context, considering three levels of pronunciation development: phonological 
awareness, perception, and production. Our pre/posttest results indicate significant 
improvements in the participants’ awareness and perception of the English past -ed, 
but no improvements in production (except for /id/). These findings corroborate our 
hypothesis that GT’s speech capabilities can be used as pedagogical tools to help 


learners acquire the target pronunciation feature. 


Keywords: text-to-speech synthesis, automatic speech recognition, L2 
pronunciation, Google Translate. 


1. Introduction 


The limited amount of time is one of the problems that teachers and second 
language (L2) students face in language classrooms (Collins & Munoz, 2016). Such 
constraint deprives students of receiving sufficient linguistic input (e.g. listening 
to the L2) and producing output (e.g. speaking with others). One of the areas in 
which such time restrictions can negatively impact learning is L2 pronunciation, as 
it requires hours of practice (Everly, 2019) and it must address at least three stages 
of phonological development: phonological awareness, perception, and production 
(Celce-Murcia, Brinton, & Goodwin, 2010). To address this limitation and at the 
same time encourage students to practice, teachers often ask their students to engage 
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in self-directed, out-of-classroom activities in the form of homework assignments, 
using technologies such as TTS and ASR. TTS and ASR can be used as effective 
pedagogical tools for pronunciation (Liakin, Cardoso, & Liakina, 2017), as they 
engage learners with a wide range of out-of-classroom activities, including the 
promotion of input (TTS) and output (ASR) practice. Their use also contributes 
to fostering learners’ autonomy beyond the walls of classrooms (Van Lieshout & 
Cardoso, 2022). 


However, TTS and ASR have only been studied separately (for an exception, see 
Van Lieshout & Cardoso, 2022). Therefore, little is known about the feasibility of 
using TTS and ASR combined (as found in GT) as pedagogical tools, nor do we 
understand what happens when learners are asked to use the technology on their 
own, outside of their classrooms (e.g. to complete homework assignments). To 
address this gap, this study asked the following question: using GT’s TTS and ASR 
capabilities in a teacher-guided semi-autonomous context, can English learners 
acquire the pronunciation of English past -ed allomorphy (/t/, /d/, /id) in terms of 
awareness, perception, and production? 


2. Method 


Following a pretest-posttest research design, 20 intermediate-level English as a 
second language students (age: 30-40; L1: Farsi) were recruited to participate 
in this two-hour one-shot study. Participants took a set of pretests on awareness, 
perception (aural discrimination), and production of the past -ed allomorphy. Two 
instruments were adopted for each measure, for a total of six tests. 


For awareness, participants were asked to answer four open-ended questions to 
determine whether they knew how past -ed is pronounced (Test 1), and then asked 
to match a set of past tense verbs based on how they believed their inflections are 
pronounced (i.e. they had to choose used for /d/, added /id/, and asked /t/; Test 2). 
For perception, two tests required participants to aurally discriminate between past 
and non-past constructions (Test 1) and then identify the -ed allomorph among 
those selected as ‘past’ (Test 2). Finally, for production, participants engaged in 
two audio-recorded tests: they read aloud a list of words containing past -ed forms 
(Test 1), followed by a less controlled speaking task in which they role-played an 
‘interview’ with one of the researchers. 


After completing the pretests, participants watched a video containing 
instructions on how the regular past tense is formed (without any information 
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about pronunciation), and how to use GT’s speech features. For the treatment, 
which emulated the completion of a homework assignment, participants engaged 
in activities that encouraged them to listen (via TTS; e.g. they copied and pasted a 
text into GT, listened to it, and then filled-in blank spaces) and produce the target 
forms (via ASR; e.g. they read aloud past -ed forms into GT and then verified 
if the intended form was transcribed correctly). At the end of the experiment, 
participants completed six posttests (similar to the pretests, but containing 
randomized items). 


3. Results 


Using t-tests, the participants’ performance was analyzed in terms of phonological 
awareness, aural discrimination (perception), and production. 


Awareness: The results from the two awareness tests indicate that the pedagogical 
use of GT’s speech capabilities contributed to the participants’ development of 
awareness of past -ed allomorphy (Table |), particularly regarding /id/ and /t/ (but 
not /d/; Table 2). 


Table 1. Awareness #1 
Pretest Posttest 
M/4 SD M/4 SD t-test 
Total 3.30 0.80 3.80 0.41 *_3.24 
*p < 0.05 
Note. M/4=Mean (out of 4); SD=Standard Deviation. 
Table 2. Awareness #2 
Pretest Posttest 
M/7 SD M/7 SD t-test 
/d/ 3.90 1.45 4.35 1.14 -1.53 
/id/ 2.85 1.04 4 1.02 *_3.81 
/t/ 1.55 0.99 2.3 1.03 *_2.44 
Total /21 8.30 1.69 10.65 2.00 *_5.17 
*p < 0.05 


Perception: Findings from both tests indicate that the GT-based treatment helped 
the participants improve their perception of -ed allomorphy in general (Table 3) 
and across the three allomorphs (Table 4). 
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Table 3. Perception #1 
Pretest Posttest 
M/15 SD M/15 SD t-test 
Total 10.65 1.98 12.25 0.91 *-5.14 
*p < 0.05 
Table 4. Perception #2 
Pretest Posttest 
M/5 SD M/5 SD t-test 
/d/ 1.65 0.81 2.4 0.68 *_4 26 
/id/ 2.4 0.68 3.4 0.59 *-7.96 
/t/ 0.9 0.71 1.3 0.66 *-2.18 
Total /15 4.95 1.54 7.1 1.25 *.9.73 
*p < 0.05 


Production: Finally, the results for production in both tests (Table 5 and 
Table 6) indicate that the proposed pedagogical intervention had no effect on the 
improvement of -ed among the participants. When considered in isolation, only the 
production of the /id/ allomorph was positively affected by the treatment. 


Table 5. Production #1 

Pretest Posttest 

M/20 SD M/20 SD t-test 
/d/ 7TA5 2.30 6.90 2.38 1.31 
/id/ 5.05 1.96 5.65 2.18 *_2.85 
It/ 3.60 137 3.65 1.69 22 
Total /60 15.80 5.54 16.20 5.54 -1.28 
*p < 0.05 
Table 6. Production #2 

Pretest Posttest 

M/10 SD M/10 SD t-test 
/d/ 3.70 1.45 3.55 1.54 0.90 
/id/ 29 1.19 2.95 1.19 *_2.63 
/t/ 0.90 0.85 0.95 0.82 -0.32 
Total /30 7.15 3123 TAS 2.99 -1.24 


*p < 0.05 


Hamidreza Khademi and Walcir Cardoso 


4. Discussion and conclusion 


This study examined the use of GT and its embedded speech features (TTS, 
ASR) as L2 pronunciation learning tools to find out whether English learners can 
acquire the pronunciation of English past -ed in terms of awareness, perception, 
and production in a teacher-guided semi-autonomous context (conceptualized 
as a homework assignment). Our findings suggest that there were significant 
improvements in both the participants’ awareness and perception of English past 
-ed allomorphy, thus confirming the hypothesis that the pedagogical use of GT 
and its speech capabilities can help learners acquire the target pronunciation 
feature in these two first stages of L2 pronunciation development (Celce-Murcia 
et al., 2010). However, in terms of production, no significant progress was 
observed, except for /id/. 


Considering the above mentioned findings, the present study offers some 
important pedagogical implications. The most important one is that it has 
demonstrated that learners can acquire certain aspects of L2 pronunciation (e.g. 
phonological awareness and perception of past -ed allomorphy) when engaged 
in teacher-initiated semi-autonomous activities such as those that characterize 
homework assignments. Via technologies such as GT and its speech capabilities, 
teachers can mitigate the time limitation that precludes them from focusing on 
pronunciation instruction, and consequently extend the reach of their classrooms 
to an environment that has the potential to provide input that is abundant and 
varied (via TTS), with ample opportunities for production practice and feedback 
(via ASR). 


Despite these encouraging results, there are limitations that need to be addressed 
in further investigation. The first main limitation relates to the two-hour duration 
of the study, as research has shown that English learners need more than two 
hours of practice to acquire past -ed (Cardoso, 2018). Although we recognize 
this limitation, we also acknowledge that there is ecological validity in our 
pedagogical implementation, as it simulates a common learning situation in 
which students are asked to complete homework assignments on their own, with 
accessible technology such as GT. Another limitation is the number (n=20) and 
uniformity of the target population (Farsi L1 native speakers). Although our 
findings support the hypothesis that GT can help these learners improve their 
pronunciation skills, it is not clear whether these findings are generalizable to the 
larger population of English learners. 
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Linggle 2.0: a collocation retrieval system 
with quality example sentences 


Shu-Li Lai', Jason Chang’, Kuan-Lin Lee’, and Wei-Chung Huang* 


Abstract. Linggle is a pattern-based referencing tool that assists in collocation 
learning. In this ongoing project, we aimed to improve its performance further. First, 
many of the example sentences are long and difficult for students to understand, so 
we used a machine learning method and trained a classifier to help select dictionary- 
like example sentences. Second, we created a database of 60,270,000 sentences from 
4C, S2ORC, and VOA Learning English. We also included Google books for real- 
time supplements. Then, we applied the classifier to select good example sentences 
from the database for display. We also limited the number of example sentences 
displayed for search results to improve users’ experiences. Two classes of English as 
a Foreign Language (EFL) college students (N=51) were invited to use the enhanced 
tool and filled out a questionnaire. The results showed that the students were positive 
about Linggle’s new interface and the quality of the example sentences. We expect 
that more EFL learners will benefit from the tool. 


Keywords: collocation tool, example sentences, machine learning, interface. 


1. Introduction 


For EFL writers, dictionaries are essential tools that help transform learners’ ideas 
into language. However, it is not always easy to find information on collocation and 
lexical grammar in a dictionary. In recent years, corpus tools have gained attention. 
Research has shown that corpus tools can supplement dictionaries to provide 
information on collocation and lexical grammar (Lai & Chen, 2015). Learners 
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can generate rules from corpus examples and learn how words and phrases are 
used in context (i.e. data-driven learning, see Boulton, 2017). However, observing 
corpus data can be time-consuming (Yoon, 2016), and it is especially challenging 
for students with lower English proficiency. 


The Linggle collocation retrieval system is a web-based service that generates 
and displays information through recurring word patterns (https://linggle. 
com/). After typing keywords and simple syntax commands, the system shows 
information on collocation explicitly. Linggle has been through several revisions 
since the first prototype was developed (Boisson et al., 2013). In an empirical 
study that we conducted on the earlier version of Linggle, students found the tool 
easy to use and very efficient in helping them find collocation information (Lai 
& Chang, 2020). 


However, there were some problems and room for improvement. Many of the 
example sentences are long and difficult to understand in the current version. 
Students also complained that there were too many example sentences, which 
overwhelmed them. Authentic sentences are often long and more complicated. In 
this ongoing project, we aimed to improve the quality of the example sentences to 
make them easier for language learners to read. Second, we also aimed to improve 
the interface so users would not be overwhelmed. Finally, we recruited two classes 
of EFL college students to evaluate the enhanced tool. 


2. Method 


2.1. Procedure 


The Internet has plenty of texts, so it is easy to find sentences containing certain 
patterns or collocations. However, some sentences may not be appropriate for 
language learners because they are too difficult to understand. Inspired by the 
GDEX model (Kilgarriff et al., 2008), we originally planned to take a rule-based 
approach to extract sentences from the web. For example, limit the sentence 
length, eliminate non-alphanumeric characters, and exclude sentences containing 
blacklist words. However, we found that sentences fitting the rules were not 
necessarily good example sentences. Some were still difficult to comprehend 
and contained difficult words. Our project took a different approach: we used the 
machine learning method and trained a classifier to help select dictionary-like 
examples automatically. 
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We used the pre-trained BERT model to extract features for the classifier. Positive 
data included 140,000 example sentences from the Cambridge dictionary and 
negative data comprised a random sample of 140,000 example sentences from 
Wikipedia, as those sentences are typically not as good as sentences from the 
learners’ dictionaries for language learning purposes. The two sets of sentences 
were combined to create a dataset for training, developing, and testing. For training 
and development purposes, we randomly split the dataset into a training set (80%), 
a development set (10%), and a test set (10%). After training, the test set was 
used to evaluate the performance of the model. The average accuracy was 0.95, 
indicating that the model was capable of distinguishing between good (dictionary- 
like) and bad (not dictionary-like) sentences. Then, we applied this model to a 
collection of open-source corpora to create an example dataset for retrieving and 
presenting quality example sentences for words and phrases in our enhanced 
collocation retrieval system. 


2.2. Corpus 


We used 60,270,000 sentences from four sources to compile our dataset of example 
sentences: C4, S2ORC, VOA Learning English, and Google books. We sampled 
about 30 million sentences from the subset of ‘realnewslike’ in Google’s C4, a 
colossal corpus of cleaned web crawl data, to expand coverage of the topic; 30 
million sentences from S2ORC, an enormous corpus of scientific journals from the 
Allen Institute for AI; 270,000 sentences from VOA Learning English; and Google 
books for real-time supplement sentences if there were any phrases not covered by 
the previous three corpora. 


2.3. Interface 


The major modification was the restriction of example phrases for each collocation 
combination. Using ‘to gain knowledge’ as an example, the upgraded 2.0 version 
of the system presented five example phrases to users first. If the user clicked on 
‘show more’, the system displayed five additional sentences, as shown in Figure | 
and Figure 2 below. 


2.4. Questionnaire 


We conducted a small-scale study with 51 EFL college students to collect their 
initial feedback. The participants’ proficiency levels ranged from A2 to B2. 
Three class meetings were arranged to orient the students to learn the concept of 
corpus, collocation, and the syntax commands of Linggle. A six-point Likert scale 
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questionnaire was designed to understand their experiences in using the upgraded 
version of the system and to elicit comments. 


Figure 1. Example sentences for ‘to gain knowledge’ 


to v. knowledge | | 


Phrases % Count 
to share knowledge 11.4% 84,000 
to gain knowledge 10% 74,000 


¢ You read a recipe to gain knowledge about baking rhubarb pie. 
© Do researchers apply a special kind of work process or procedures to gain knowledge? 


© Credit is a pragmatic good that we gain through believing well enough to gain 
knowledge. 


¢ While you broaden your professional circle, leverage the opportunity to gain 
knowledge rapidly. 


It is fundamental to gain knowledge regarding strategies that can potentiate neural 
regeneration. 


~ Show more 


Figure 2. More example sentences for ‘to gain knowledge’ 


to gain knowledge 10% 74,000 


¢ You read a recipe to gain knowledge about baking rhubarb pie. 

¢ Do researchers apply a special kind of work process or procedures to gain knowledge? 

¢ Credit is a pragmatic good that we gain through believing well enough to gain knowledge. 

¢ While you broaden your professional circle, leverage the opportunity to gain knowledge rapidly. 


¢ Itis fundamental to gain knowledge regarding strategies that can potentiate neural regeneration. 


The FSFI questionnaire allows us to gain knowledge about female sexuality and the factors 
affecting it. 


This Internet-assisted learning facility has given children the opportunity to gain knowledge on 
the go. 


Third, students defined career communities as opportunities to gain knowledge they could not 
find online. 


The discussions with experts gave an opportunity to gain knowledge of the target group's and 
their stress. 


Some trainees thought they should study these problems more extensively to gain knowledge for 
future cases. 


- Show less 
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3. Results and discussion 


Table | shows the results of the participants’ initial experiences with the improved 
system. In general, students found Linggle highly efficient. It helped them with 
most of the collocation problems (M=5.3). Students also found Linggle easy to 
use (M=5.3) and the syntax commands are easy to learn (M=5.2). The findings are 
consistent with the previous Linggle study (Lai & Chang, 2020). 


Table 1. Questionnaire results on efficiency and usability of the tool 


Item AVG 
1 Linggle helped me with most of the collocation problems. 53) 
2 Linggle quickly helped me solve collocation problems. 5.1 
3 I find it easy to use Linggle to look up collocations. 5.3 
4 The Linggle syntax commands are easy to learn. 3.2 


Table 2 shows the participants’ perceptions of the new interface and the information 
the system provided, including the collocation information and the example 
sentences for each collocation. The students were pleased with the interface, which 
was Clear, intuitive, and easy to use. They also believed that the enhanced version 
offered a reasonable number of example sentences (M/=4.5) and these sentences 
were not too hard (M=4.7). Observing corpus examples is never easy. Learners 
often experienced some frustration when observing corpus examples (Yoon, 2016). 
In this project, the example sentences were selected by the classifier we trained. 
The dictionary-like sentences were easier for the EFL learners to comprehend and 
would reduce learners’ cognitive load. 


Table 2. Questionnaire results on interface and richness of example sentences 


Item AVG 
1 Linggle’s interface is user-friendly. 5.4 
2 I like how Linggle displays its search results. 3:2 
3 Linggle offers a lot of information on patterns and collocations. 5.2 
4 The examples that Linggle gives are not too hard or too easy. 4.7 
5 Linggle offers a reasonable number of example sentences. 4.5 
4. Conclusion 


This paper reported the improvements made to the collocation search engine 
Linggle. We used the machine learning method and trained a classifier to help 
select good example sentences automatically. Initial results were quite positive. We 
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also enhanced the interface to avoid overwhelming users. If a corpus tool is well- 
designed, it can be very beneficial. Our enhanced version will provide EFL learners 
a quick way to obtain collocation information. 
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Edutainment: assessing students’ perceptions 
of Kahoot! as a review tool in French L2 classes 


Oneil N. Madden! 


Abstract. Gamification is a method used to engage and motivate students virtually 
during the coronavirus pandemic. This paper reports on the use of Kahoot! as a 
review tool in French as a foreign language (L2) classes at Northern Caribbean 
University (NCU), Jamaica, as a means of formative assessment. Using the 
exploratory approach, it seeks to highlight students’ perceptions of this platform 
to enhance teaching and learning. Twenty-one students of both genders between 
the ages of 18 and 35 responded to a post questionnaire, after participating in 
two or more games of Kahoot!. Data was also collected through semi-structured 
interviews and observations recorded by the instructor. Preliminary findings 
show that Kahoot! helps most students (95%) to improve their French, in terms 
of learning and/or reinforcing new vocabulary or previously taught concepts, as 
well as pronunciation. All the participants highlighted that Kahoot! is fun and 
interactive, and playing the game serves as a motivation for them to revise their 
already acquired knowledge. However, the countdown feature can provoke stress 
and affect the thought process. 


Keywords: Kahoot!, gamification, perception, French as a foreign language, 


Jamaica. 


1. Introduction 


In response to the coronavirus pandemic, “many countries around the world closed 
schools, colleges and universities to halt the spread of the virus” (OECD, 2020, 
p. 2). Consequently, many systems adopted virtual learning at an unprecedented 
level by exploiting technological devices and the Internet. However, despite 
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the subsequent return of face-to-face classes in many universities worldwide, 
most Jamaican institutions have maintained online classes. As both students 
and instructors must adjust to new and sudden pedagogical dynamics, many 
educators, including L2 instructors, have to find creative and innovative ways 
to diversify their classes so that students can be engaged and motivated to learn 
(Madden, 2022). 


From a psychological perspective, motivation is a necessary personality factor 
that L2 learners need to achieve meaningful production. Gardner (2010) posits 
that a motivated student engages in the relevant activities, expends efforts, persists 
in activities, attends to tasks, shows desire to achieve the objective, and enjoys 
the activities. Gamification is one approach that fosters that motivation. Kapp 
(2012) defines gamification as “using game-based mechanics, aesthetics and game 
thinking to engage people, motivate action, promote learning, and solve problems” 
(p. 10). Manzano-Leon et al. (2021) highlight that gamification fosters greater 
motivation, academic performance, and commitment in students while Buckley 
and Doyle (2016) indicate that gamification reinforces extrinsic motivation and 
improves intrinsic motivation. 


2. Kahoot! 


Kahoot! is a free game-based digital platform, which was developed in Norway 
(https://kahoot.com/). Research shows that Kahoot! is a platform that motivates 
and activates students’ learning and engagement (Wang & Lieberoth, 2016) 
because it can test their knowledge, reiterate important concepts, and help 
them retain information. Plump and LaRosa (2017) add that Kahoot! enhances 
classroom dynamics, as the game gives students immediate feedback on their 
performance, and instructors can design and incorporate teaching activities to 
address any weaknesses in students’ responses to the questions. 


Thousands of Kahoot! games have been played across countries and institutions, 
and numerous research conducted; however, there is a dearth of documented 
experiences in the Jamaican context. Wanliss (2021), a teacher-trainer of French, 
recorded testimonies of Jamaican pre-service L2 teachers, who stated that Kahoot! 
was useful during their virtual teaching practicum. This scarcity of local data will 
benefit from this paper, which reports on the use of Kahoot! as a review tool in 
French L2 classes in a higher education context. Using the exploratory approach, 
the researcher seeks to ascertain students’ perceptions on the use of Kahoot! as well 
as some benefits of gamification in the L2 classroom. 
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3. Method 


This paper reports on the use of Kahoot! in French L2 classes at NCU, Jamaica? 
throughout the academic year 2021/2022, among undergraduate learners (N=21), 
both male (N=7) and female (N=14), between the ages of 18 and 35 years. The 
students were from different humanities and hospitality majors and followed 
beginners or intermediate French level courses as part of their programme 
requirements. 


During the 15-week semester beginners’ course (spring semester), the researcher 
played three Kahoot! games and four games in the intermediate course at different 
intervals to review/reinforce unit content. Each game comprised between ten 
and 20 questions on syntax, vocabulary, idiomatic expressions, and culture or 
interculturality (see example in Figure |). The questions were tailored based on the 
challenges faced by the students during the lessons. After each question-response 
period, the instructor provided clarity on both the correct and incorrect responses. 


Figure 1. Sample of Kahoot! questions 


Quiest-ce que vous le week-end ? {le présent) & e q the past participle of "Prendre” Ei 
+ noe 


0 
réponse 


ws & kahoot.it Code PIN du jeu 6812469 2/20 @ kahootit Code PIN du jeu : 1496775 


cy 


ty) 


réponse 


The data collection included semi-structured interviews and observations 
conducted by the instructor throughout the course (before, during, and after 
playing Kahoot!). The primary questions surrounded students’ linguistic (all 
skills) and cultural gains after each session. Additionally, a questionnaire was 
given towards the end of the spring semester, which consisted of eight closed- 
ended and five open-ended questions. The former questions primarily accounted 
for demographic data, while the latter focused on ways in which Kahoot! helped 
to improve their L2 skills, their perception of Kahoot!, and their thoughts on 
gamification in L2 education. 


2. Thanks to all students from the Beginners 2 and Intermediate | and 2 French courses for participating in this study. 
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4. Preliminary findings and discussion 


In the questionnaire, 72% of the respondents declared that before taking their 
French course, they had prior knowledge of and exposure to Kahoot!, while 28% 
had no familiarity with it. Kahoot! was used in other courses to aid revision, and 
in social and cultural contexts for entertainment as well as competitive purposes. 


Concerning the use of Kahoot! to improve students’ competence in French, 62% 
of the participants indicated that the tool helped to improve their level, 33% 
confirmed that it helped somewhat, while only 5% noted that it did not help at all. 
The majority of the respondents highlighted that the games played helped them to 
learn and/or reinforce new vocabulary and pronunciation, as well as previously 
taught concepts (see Figure 2). 


Figure 2. How Kahoot! improves French L2 competence 


Learn and/or reinforce new 
vocabulary 


18 (85.7%) 


Learn and/or reinforce 
pronunciation 


Learn and/or reinforce concepts 16 (76.2%) 


3 (14.3%) 


None of the above 1 (4.8%) 


An extrapolation of the data from the instructor’s semi-structured interviews 
and observations, as well as students’ self-reported gains from the questionnaire, 
also corroborate the above-mentioned findings (see Excerpt 1, supplementary 
materials). 


Other advantages highlighted from the different data sources include metacognitive 
and interpersonal development, as students employ different strategies to respond 
to the questions, and they display great sportsmanship, encouraging and supporting 
each other throughout the games. However, some students underscore that the 
countdown feature can provoke anxiety, while for others, technical and connectivity 
issues can be a downfall (see Excerpt 2, supplementary materials). 


Data from the questionnaire suggests that the incorporation of games in the L2 
classroom creates a fun, interactive, and competitive environment, which makes 
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the context not so strict. Additionally, games tend to grab students’ attention and 
motivate them to learn. Competition also encourages some students to revise 
their notes if they want to be ranked highly in the games. Moreover, the use of 
games helps students to recap learned content and identify their strengths and 
weaknesses (see Excerpt 3, supplementary materials). For these reasons, 86% of 
the respondents suggested that Kahoot! should be played after each learning unit, 
while 14% recommended once monthly. Seventy-one percent indicated that getting 
prior notice of the game would motivate them to revise, while 29% said it would 
somewhat motivate them. 


An initial analysis of the findings supports Wang and Lieberoth’s (2016) position 
that Kahoot! fosters learners’ motivation, reinforces already taught concepts, and 
promotes information retention. Additionally, Kahoot! as gamification in general 
shifts classroom dynamics (Plump & LaRosa, 2017) and brings more diversity 
to an often-monotonous setting. However, in order for gamification to yield such 
results, the questions have to be carefully designed to prompt students to utilise 
their metacognitive strategies to produce meaningful output. Additionally, the 
instructor must spend time to clarify any misconceptions or questions that students 
may have. 


5. Conclusions 


This study suggests that there is a correlation between gamification and students’ 
motivation and involvement in L2 classes. All the participants in this study 
reported that gamification is fun and engaging, and 95% of them confirmed that 
the use of Kahoot! enhanced their French language and cultural skills. 


Kahoot! promotes student learning and reinforces the acquisition of new 
vocabulary, previously taught concepts and grammatical structures, and 
pronunciation. It also aids in the development of metacognitive and interpersonal 
skills. However, technical and connectivity issues can arise, and timing plays an 
important role. Gamification should be carefully designed and executed to yield 
meaningful output. 


6. Supplementary materials 


https://research-publishing.box.com/s/grpagqq8socklgfvnzef39fp456 1 c8qs 
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Abstract. Globalisation amplifies the need to improve Intercultural Communicative 
Competences (ICC). However, telecollaborative cross-cultural communication may 
be affected by different factors such as morals, values, and differences in viewpoints, 
as observed in numerous European and North American projects. Still, there is a 
dearth of experiments from the Anglophone Caribbean’s stance. Therefore, this 
paper seeks to highlight how religious ideologies affect international communication 
in ClerKing — a Franco-Jamaican telecollaborative project that occurred in different 
phases over a three-year period with learners of English from Clermont Auvergne 
University (UCA) and learners of French from Shortwood Teachers’ College (STC) 
and the University of the West Indies (UWI), Mona. Using the exploratory approach, 
pre- and post-project questionnaires, learning and reflective journals, and different 
multimodal exchanges were analysed. Preliminary findings suggest that religion 
was a major theme highlighted in all phases of the project, leading to challenged 


communication and somewhat negative perception of the target culture. 
Keywords: Intercultural communicative competence, religion, telecollaboration, 
ClerKing, Jamaica. 

1. Introduction 

The need for people to cultivate and demonstrate ICC and global competence 


is more demanding in this interconnected world. Globally competent people 
possess the capacity to “examine local, global and intercultural issues, understand 
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and appreciate different perspectives and world views, interact successfully and 
respectfully with others, and take responsible action toward sustainability and 
collective well-being” (OECD, 2018, p. 4). Foreign language educators have used 
telecollaboration to help learners to develop their cross-cultural skills. In fact, 
several thousand telecollaborative projects have been implemented over the past 
two decades but mainly in European and North American contexts. There remains 
a dearth of perspectives from the Caribbean’s standpoint, including Jamaica, 
which is relatively new to incorporating telecollaboration in educational curricula. 
Not many Jamaican students participate in study exchange programmes due to 
financial constraints or lack of opportunities within their degree programmes. 
Thus, telecollaboration is a cost-effective way to get students to interact with other 
cultures. 


The literature shows that numerous advantages are associated with telecollaboration, 
such as improvement in cultural awareness and linguistic competence. However, 
O’Dowd and Ritter (2006) attribute ‘failed’ communication to four domains: 
individual, classroom, socio-institutional, and interactional. In addition, Madden 
and Ashby (2021) note that the heterogeneity of cultures and divergence in cultural 
perspectives, morals, and values can provoke interactional friction. This happens 
within a ‘contact zone’ (Pratt, 1991), where cultures meet, clash, and grapple with 
each other. 


Helm (2015) found that participants prefer to focus on ‘safe’ topics, but 
circumventing conflict in exchanges may, however, falsify the reality of intercultural 
encounters. In fact, exploring sources of disparity over serious issues, including 
religious beliefs, can be revelatory, a rich cultural point (O’ Dowd, 2011). The aim 
in exposing students to potential conflict is not for everyone to agree in the end, but 
for students to understand that otherness exists. 


Using the exploratory approach, this paper seeks to share insights on the extent to 
which religious ideologies affected the outcome in the ClerKing telecollaborative 
project. 


2. Method 


ClerKing is a Franco-Jamaican project that spanned over three years (2018-2020) 
between Applied Foreign Language learners of English from UCA, France, and 
learners of French from STC and the UWI, Jamaica. French students took the 
course Open Learning Project and were B2-Cl1 level on the Common European 
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Framework of Reference for Languages, while Jamaican students took the course 
Conversation French and had an A1-B2 level. There were four phases, ranging 
from seven to ten weeks, depending on the semester, and 121 participants of mixed 
genders, between the ages of 18 and 33 participated. Students were paired/grouped 
based on their profiles submitted ahead of the project, which included linguistic 
and cultural background, areas of studies, likes and dislikes, gender, age, and place 
of birth and residence. They discussed different cultural and intercultural topics 
weekly via different media (videoconferencing platforms, WhatsApp) to improve 
linguistic, cultural, and intercultural competences. They also completed specific 
tasks such as learning journals and reflective journals to document their experiences 
and gained knowledge. Table | provides more details into the breakdown of the 
project. 


Table 1. Breakdown of the ClerKing telecollaborative scenario 


Language of In Phases 1-3, the languages were imposed — French for half 
exchange an hour and English for the other half to facilitate practice. The 
UWI participants in Phase 4 had an A1/A2 level in French, 

so they had the option to choose which language to use. 


Topics discussed | Getting to know you, geography, historical moments, culture/ 
customs/stereotypes, politics, regional languages, national 
celebrations, literature, multiculturalism, taboo subjects. 

The topics were imposed in Phase 1, while participants 

had a say in choosing two of the topics in Phases 2-4. They 
also chose which ‘taboo’ topic they wanted to discuss. 


Tasks Final: Choose one of the topics discussed and create 
a brochure or a YouTube video to share the content 
learned from discussions with your partner/s. 


Communication | WhatsApp was used for text and voice messages in between, while 
platforms Skype and Zoom were used for longer video conversations. 


The data collection includes pre- and post-project questionnaires (see 
supplementary materials), journal entries, and the different exchanges had. 


3. Findings and discussion 


Preliminary findings show that religion or religious views played a critical role 
in determining the overall outcome of the ClerKing project. An extrapolation of 
data across all phases seems to link students’ morals and values to their religious 
stance, whether they are practising believers or not. This was observed in their 
discussions on controversial topics such as same-sex unions, abortion, and 
Valentine’s Day. Forty-five per cent of Jamaican students referenced the Christian 
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Bible, stating that homosexuality is a sin, and those who practise it will have no 
part in God’s kingdom. On the other end, 60% of Clermontois students in their 
defence indicated that the problem is not about homosexuality; it is just that some 
people are close-minded. One Jamaican student described the conversation as 
‘blasphemous’ before exiting the WhatsApp group chat. Similarly, on the matter 
of abortion, the same Jamaican participants cited the Bible to justify their stance 
for being pro-life, by referring to the commandment that says, ‘thou shalt not 
kill’. Conversely, 30% of their Clermontois counterparts spoke of feminism and 
the right for a woman to do as she pleases with her body. Regarding Valentine’s 
Day, some Jamaican participants made similar pronouncements that they do 
no celebrate this event because of the alleged view that Saint Valentine was a 
homosexual. Most of these exchanges were halted because, on the one hand, 80% 
of the Clermontois students felt that their Jamaican peers were intolerant and 
unwilling to view intercultural issues from the other person’s perspective. On the 
other hand, 85% Jamaican participants felt like their religious beliefs were being 
mocked. Efforts were made to address the concerns in a mutual Zoom call but 
most participants were opposed to this meeting. 


In other instances, personal religious schedules affected many of the exchanges. 
Amid the time difference (six or seven hours, depending on the semester), all the 
Jamaican participants in Phases | and 2 had to go to daily morning devotional 
exercises, which were part of their university curriculum. Likewise, 75% of them 
had mid-week religious services at church on the weekend. At times, they would 
agree to a meeting time with their Clermontois partners, but they either showed 
up late, cancelled at the last minute, or were absent due to prioritising religious 
activities over the telecollaborative project. 


Sixty per cent of Clermontois participants found it appalling that religious education 
is integral in most Jamaican schools, given that they are accustomed to /aicité. One 
participant in Phase 3 summed up this idea by saying, 


“T learned that religious teaching was important, even going to say prayers 
in the morning before the school day begins. Being a French atheist, any 
religious concept is distant to me, so when I am told that you can be taught 
religious principles at school, it is always surprising to me”. 


The findings indicate that religious ideologies contributed to failed communication 
in telecollaboration, touching the individual, socio-institutional, and interactional 
dimensions described by O’ Dowd and Ritter (2006). Participants operated in their 
‘contact zones’ (Pratt, 1991) on intercultural issues, which led to intercultural 
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friction (Madden & Ashby, 2021) due to a failure to understand and appreciate 
others’ perspectives and engage in open interactions (OECD, 2018). 


4. Conclusions 


This study suggests that morals and values, which are influenced by religious 
ideas, can lead to a lack of openness in cross-cultural communication. While this 
may lead to challenged communication, it is necessary for curricula to continue 
to facilitate intercultural exchanges, whether physically or virtually, as the 
development of global competence requires cultures to interact with each other 
to cultivate certain skills such as tolerance and curiosity. 
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A telecollaborative study of university students 
in Spain and Sri Lanka using the Soqqle video app 


Marni Manegre! and Piyumi Udeshinee?” 


Abstract. This study addresses interculturality and Intercultural Competence (IC) 
by connecting university students in Spain and Sri Lanka through the use of the 
Soqqle app, an education-based video app. The students were asked to create five 
videos independently using English as a lingua franca and upload each video to 
the app related to the assigned tasks. The goal of this study is to determine whether 
creating videos increases the IC of the students and whether the video creation 
activities enhance the students’ English as a Foreign Language (EFL) listening 
and speaking skills. The students were given a pre-questionnaire at the onset 
and post-questionnaire at the conclusion of this study. The results show that the 
students increased in their cultural knowledge from the pre- to post-questionnaire. 
Additionally, the students reported that their Foreign Languages (FL) skills 
increased, and they generally enjoyed participating in this study. 


Keywords: virtual exchange, telecollaboration, mobile assisted language learning, 
social media language learning. 


1. Introduction 


Interculturality, when introduced in a classroom, often relies on the biased 
and subjective comparisons of different cultures (Li & Dervin, 2018). When 
interculturality is introduced, students not only learn what is culturally 
appropriate, despite any biases, but they also develop an understanding of how 
to use language to build relationships across cultures (Belz, 2003; Ferreira, 
Bezanilla, & Elexpuru, 2018; Kramer Moeller & Nugent, 2014). In addition 
to interculturality, which is a comparison of cultures, IC is often introduced in 
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classes to teach students how to have meaningful interactions with those from 
different backgrounds. There are several approaches to how teachers introduce 
online communication in a classroom with the goal of increasing IC (O’ Dowd, 
2018). This study incorporates Deardorff’s (2006) model of IC and aims at 
answering what it means to successfully communicate and interact with those 
from different cultures (Deardorff, 2006, 2011). In this model, there are three 
main components that can be assessed, which are the attitudes of the individuals, 
which includes respect, openness, and curiosity to discover, knowledge and 
comprehension, which includes cultural self-awareness, cultural knowledge, 
and sociolinguistic awareness, and finally skills, which is the ability to observe, 
listen, and evaluate. The current study focuses primarily on assessing the 
knowledge and comprehension, along with the skills that the students acquired 
in a telecollaborative virtual exchange that connected students from the Catalan 
region of Spain with students in Sri Lanka. Additionally, the attitudes of the 
students were assessed, but mainly regarding the enjoyability of the project. 
Since students between Spain and Sri Lanka are not often paired in a virtual 
exchange, we were interested in observing how the students advanced in their 
knowledge of the foreign culture, the advancement of the students’ skills in using 
English as a FL, and their overall attitudes from participating in this project, such 
as their level of enjoyment. Thus, the following research questions were asked. 


¢ Will using the Soqqle video application to create short videos as part of 
this telecollaboration project increase the cultural knowledge and IC of the 
participating students? 

¢ Will the participation in this project by creating and watching videos of 
their peers and partners in the foreign country increase their EFL speaking 
and listening skills? 

¢ Will the students enjoy creating videos in English and participating in the 
project? 


2. Method 


2.1. Participants 


There were 56 students enrolled in the English degree program at a university in the 
Catalan region of Spain and 54 students enrolled in the business English program 
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at a university in Sri Lanka who participated in this study. All the participating 
students have a level of English between A2 and C2 on the Common European 
Framework of Reference for Languages scale. 


2.2. Materials 


This study employed the use of the Soqqle. Soqqle is an education application 
that allows students to create videos as part of their homework assignments. This 
app is secure, and the students and teachers require a class code to upload and 
watch the videos. The app was built for asynchronous learning, as there is no live 
stream. Videos are uploaded to the app, and it functions like other social media 
apps allowing peer-based learning, since the students can like their peers’ videos 
and leave comments. 


2.3. Procedure 


The students filled out a pre-questionnaire that assessed their knowledge of the 
foreign country they would be working with. Over the course of the autumn 
semester in 2021, the students uploaded five videos based on the assigned theme. 
They were required to watch and comment on the other videos with instructions 
on how to give valuable feedback. The themes for the videos were as follows: 
food, neighborhood, music, hobbies/sports, animals, and pets. At the end of the 
semester, the students filled out a post-questionnaire detailing their perceptions 
of language improvement, their knowledge of the foreign country, and their level 
of enjoyment. 


3. Results 


When analyzing the students’ knowledge of the foreign countries collectively, 
from the pre-questionnaire (M=3.83, SD=2.52) to the post-questionnaire (M=8.68, 
SD=2.10), the analysis shows that the results are significant, #(110)=-17.34, 
p<.001. We then examined the differences by each university using an ANOVA and 
found that there was a difference between the students at the different universities 
F(1)=22.32, p<.001. 


As can be seen in Figure | below, the students in Spain claimed to know less 
about their counterparts in Sri Lanka than their partners claimed to know of Spain. 
However, the scores of the students at both universities were similar when the 
study concluded. 
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Figure 1. Cultural knowledge of the students in Spain and Sri Lanka from the 
pre- to post-questionnaire 
10 - Country 
4 © Spain 
8- @ Sri Lanka 
6 =f 
4 
al J 
Pre-Ques Post-Ques 
Cultural Knowledge 


To analyze the perceptions of the students’ overall FL skills, FL listening skills, and 
FL speaking skills, chi-square goodness of fit tests were conducted to determine 
whether the skills were equal between the Likert options that ranged from strongly 
agree (5) to strongly disagree (1). The proportions differed for each of these 
variables, as can be seen in Table | below. The students generally reported that 
they agreed (M>3) this project increased their overall FL skills, speaking skills, 
and listening skills, as can be seen in Table 2 below. 


Table 1. Chi-square goodness of fit tests for language skills 
Location Skills 2 df p 
Spain FL General 64.000 4 <.001 
Spain FL Listening 39.714 4 <.001 
Spain FL Speaking 37.929 4 <.001 
Sri Lanka FL General 52.296 4 <.001 
Sri Lanka FL Listening 49.889 4 <.001 
Sri Lanka FL Speaking 30.630 4 <.001 
Both FL General 109.545 4 <.001 
Both FL Listening 89.182 4 <.001 
Both FL Speaking 65.182 4 <.001 
Table 2. Descriptive statistics on perceptions of skills 
Increased General FL__| Increased Listening Increased Speaking 
Mean 3.865 3.919 3.748 
SD 0.814 0.811 0.879 
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Additionally, a chi-square goodness of fit test was also completed to determine 
whether the students’ reported enjoyment when participating in this project was 
equal on the Likert scale. The proportions also differed for each variable, as 
can be seen in Table 3 below. The students generally reported that they enjoyed 
participating in this project (M>3), as can be seen in Table 4 below. 


Table 3. Chi-square goodness of fit tests for enjoyment 


Location Perceived Enjoyment | 72 df p 

Spain Enjoyed Participating | 54.893 4 <.001 
Sri Lanka Enjoyed Participating | 102.852 4 <.001 
Both Enjoyed Participating | 148.727 4 <.001 


Table 4. Descriptive statistics on perceptions of enjoyment 


Spain Sri Lanka Both 
Mean 4.339 4.618 4.477 
SD 0.837 0.757 0.870 


4. Conclusions 


While the students at the conclusion of the study generally shared the same 
amount of knowledge of the other culture, at the onset of the study, the students 
in Spain appeared to know less of Sri Lankan culture than their partners knew 
of Spain. However, the results show an increase in cultural knowledge from the 
pre-questionnaire to the post-questionnaire for both schools and this increase is 
significant. Additionally, the students from both universities reported that they 
generally felt that all three skills have increased due to their creation of videos 
in English and watching the videos of their classmates and foreign peers. Finally, 
the students generally agreed that they enjoyed participating in this project. As the 
findings support our research questions, we consider this collaboration project to 
be a success. Nevertheless, most of the data collected was based on an analysis of 
student perceptions, that is, whether they feel their skills have improved. 
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Emotional and social engagement of teenager 
and young adult students of EFL using MIM 
(Mobile Instant Messaging) 


Dunia Martinez! and Christine Appel? 


Abstract. Mobile Instant Messaging (MIM) applications have come into focus as 
potential tools to improve English language instruction, and teachers can engage 
more students from different backgrounds in English as a Foreign Language (EFL) 
classes thanks to MIM apps’ distinctive features, like WhatsApp. Most of the 
reported studies on the use of WhatsApp in the teaching of foreign languages were 
performed in university or adult learning environments. The present study explores 
how social and emotional engagement manifest themselves in a popular instant 
messaging application group (WhatsApp) used by teenagers learning EFL. A focus 
group, a Likert scale survey, and a transcript of the WhatsApp chat were the three 
main sources from which data was gathered. Members’ interaction records were 
retrieved and inductive thematic analysis was used to examine them. These findings 
suggest that WhatsApp provided communicative opportunities to all students, 
including those who did not fully engage. However, further research with a larger 
number of students in different contexts is necessary. 


Keywords: engagement, mobile instant messaging, WhatsApp. 


1. Introduction 


Technological development in mobile technology has been advancing constantly, 
improving access to information and communication almost instantly and 
anywhere. With the development of mobile technology, the opportunities for 
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Emotional and social engagement of teenager and young adult students... 


interaction among the different participants in the teaching-learning process are 
expanding (Lopez-Hernandez & Silva-Pérez, 2016). 


Young students today regularly use smartphones, social media, and instant 
messaging. Furthermore, according to recent studies (Rideout & Robb, 2018), teens 
prefer texting to talking, suggesting that MIM is their main form of communication. 
As a result of their widespread usage of mobile devices, this generation has 
developed a technological aptitude and has even begun networking and sharing 
via social media and mobile applications. MIM applications have therefore been 
identified as potential tools for enhancing foreign language learning (Andujar, 
2016; Tang & Hew, 2017). 


Although young people use MIM primarily to communicate, Tang and Hew (2017) 
claim that “we lack a comprehensive understanding of how MIM applications 
are used for teaching and learning” (p. 87). Furthermore, language teachers are 
constantly attempting to connect their students to school and learning because 
they recognise the importance of student engagement in academic achievement. 
When students are working on a task individually or in groups, they may react 
differently: some may be completely focused on the task, while others may be on 
and off. Engagement is a condition of increased attention and involvement in which 
participation is manifested not only cognitively but also socially, behaviourally, and 
emotionally (Christenson, Reschly, & Wylie, 2012). Therefore, learner engagement 
is essential for learning to be successful. 


This study explores how social and emotional engagement manifest themselves 
in a popular MIM application, WhatsApp. It aims to gain a better understanding 
of how teenage students communicate with one another and the teacher in a 
WhatsApp group. This study focuses on emotional and social engagement as social 
engagement is inextricably linked to emotional engagement, particularly among 
child and adolescent learners (Philp, Oliver, & Mackey, 2008). 


2. Theoretical framework 


Student engagement has gained much attention over the last several decades 
as instructional methods have shifted toward student-centred, constructivist 
approaches (Wright, 2011). This study draws on a student engagement model 
by Bowden, Tickle, and Naumann (2019) which incorporates four dimensions: 
behavioural, emotional, social, and cognitive engagement (Figure |), and presents 
results related to the emotional and social dimensions. 
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The social component of engagement should be considered an essential element 
of adolescent and young adult engagement (Bowden et al., 2019). It takes into 
account the connections that students make with their peers, faculty, and other 
key players in their educational process. Peer influence becomes more noticeable 
during adolescence, as teenagers spend more time with their peers and develop 
independence from their parents (Pekrun & Linnenbrink-Garcia, 2012). 


Interest, boredom, happiness, anxiety, and other affective states are all examples 
of emotional engagement and can all have an impact on how involved students are 
in their studies. In a nutshell, emotional engagement refers to both positive and 
negative responses to classmates and teachers. It is thought to have an effect on the 
student’s motivation to complete the assigned tasks. 


Figure 1. The four dimensions of student engagement (based on Bowden et al., 
2019) 


This study aims to gain a better understanding of how teenage and young adult 
students communicate with one another and their teacher in an EFL class WhatsApp 
group. With this objective, we pose the following research question: how do social 
and emotional engagement manifest itself in these WhatsApp group interactions? 
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3. Method 


This study was conducted in a real classroom environment in which WhatsApp 
was incorporated into natural educational practices. Students used the MIM service 
via the mobile app on their smartphones, and there were no mandates or constraints 
on how participants should use the app. 


The 14 participants were teenage students, aged 13-17, enrolled ina B2 EFL course 
in a language school in Catalonia (2021-2022 school year). They were all studying 
EFL as an extracurricular activity and are bilingual in Catalan/Spanish. The class 
met twice a week for one and a half hour sessions. 


The teacher was part of the WhatsApp group, and participation in the study was 
voluntary and did not affect grades. Students were encouraged to do different 
activities such as speaking tasks, both individually and in pairs, short writing 
exercises (Figure 2), and use the WhatsApp group in any way that might help 
them. 


Figure 2. Example of writing task 


CLEOUD ATLAS 


complicated plot. 0 


tell us why it is a complicated 


| haven't understand the plot at all. I've 
seen this film with a friend because it was 
mentioned in a book that we really liked. 
It's kinda hard to follow but it's still a 
good film, but so long (3h aprox) :) 


DOWNTON ABBEY 


& — 
8:05 p.m 3:41 p.m 


Omg | love this film! And the others too 


| love it too! 3:44 p.m 
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Data* was collected by means of a focus group, a Likert scale survey, and a 
transcript of the WhatsApp group (Table 1). 


Table 1. Data collected 


Dimensions of | Measuring how Type of Analysis Data Sources 
Engagement | it happened 
SOCIAL ¢ Teacher-student * Qualitative * Focus group 
interactions 
¢ (Inductive thematic | * WhatsApp group 
* Student-student analysis) chat: how they 
interactions interact, when, 


* Quantitative how often, etc. 


* Creation and 


maintenance of ¢ Number of 
relationships messages sent 
during the study 

EMOTIONAL | * Emotional reactions | * Qualitative * Focus group 


¢ Sense of belonging | ¢ (Inductive thematic | * Questionnaire 


analysis 
yi) ¢ WhatsApp group 


* Quantitative chat: number of 
emojis and words 
that express 

any emotion 


4. Results and discussion 


4.1. Social engagement 


The analysis of the WhatsApp group transcript revealed social phatic expressions 
like greetings (73 messages), holiday wishes (28 messages), and birthday wishes 
(21 messages), which helped increase students’ sense of friendliness. 


There was also self-disclosure of personal information in the chat, which helped 
them get to know each other more, and expressions of appreciation (Figure 3). 
Students reported that the WhatsApp group was useful for communicating and 
socialising with their classmates, especially when in hybrid lessons. 


3. Ethical issues and parental permission: as all students are underage, all participants and their parents signed consent forms 
for their participation in the study. 
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Figure 3. Students share personal information and expressions of appreciation 
from their classmates 


wor 


Can | post a comic strip that | made here? 


d fo 


Yes | BB here is Pare: 
| want see it 
wowww 
co 
8 here is fet 
@ here is sta! 
ee d @ awesome 
5 = al 
Sete if [Oe § 
It’s very nice!! & 
here is 
om 
<M = are i ie lee Ms 
‘if [ee se se 
ee 


= ry 


4.2. Emotional engagement 


A total of 284 emojis were used in the WhatsApp group out of 846 messages sent 
during a school year (September-June). The convenient use of emojis and GIFs 
also helped with emotional expression and added some fun to the conversation. 


Students in general reported that they had the feeling they had learned and improved 
their speaking and writing skills, although four of them mentioned that they did not 
find much difference between using WhatsApp and writing on a piece of paper. 
They saw these tasks as the same, using a different tool. 


Thirteen out of 14 students sent their speaking audio recordings directly to their 
teacher, who gave them oral feedback using a voice-recorded message. Students 
reported during the focus group that they felt less embarrassed when sending 
their speaking tasks using WhatsApp directly to the teacher because they were 
on their own and nobody in class was listening. Some stated they did not like 
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how their voice sounded; others reported that they did not want their classmates 
to notice if they made mistakes. Contrary to what some studies show (Han & 
Keskin, 2016), WhatsApp does not always reduce anxiety when speaking. 


According to the focus group discussion and the results of the surveys, the 
WhatsApp group helped students create a sense of group and connectivity and 
improved awareness of peer presence. The teacher’s participation in the group and 
her attempt to use this application in an academic setting were well-received by 
the students. 


5. Conclusions 


The present study provided an insight into how social and emotional engagement 
develops in an EFL for young learners class WhatsApp group. The WhatsApp group 
established a friendly and interactive environment, and students had a strong sense 
of belonging within the community. WhatsApp special features helped promote 
social interaction and allowed students to share positive and negative emotions 
through emojis and GIFs. 


WhatsApp provided all students, including those who did not fully engage, with 
communicative opportunities. However, further research with a larger number of 
students and in different contexts is needed. 
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Expanding design skills for language 
teachers: a corpus-based web application for 
‘language for work’ content creation in Italian L2 


Elena Michelini! 


Abstract. Many adult migrants lacking L2 linguistic autonomy cannot afford the 
time to master host countries’ languages before providing for their basic needs. 
Thus, second language training that focuses on occupational integration is at times 
more urgent than a more generally social one, as the European ‘language for work’ 
approach suggests. Given the shortage of both support sector experts in Italian L2 
classes and an appropriate variety of teaching materials for beginners, this paper 
introduces a PhD work-in-progress project proposing an IT procedure which enables 
language teachers to collaboratively design specialised multimedia content for 
beginners, thus enhancing work-related second language development. The article 
discusses the results of some preliminary surveys and argues the importance of the 
use of technology when expanding teachers’ design skills. 


Keywords: ‘language for work’, wikis, collaborative knowledge creation, teacher 
training, specialised corpora. 


1. A ‘language for work’ 


European policies require migrants to demonstrate their proficiency in host country 
languages before they may participate in all aspects of society. Yet many adult 
migrants lacking L2 linguistic autonomy cannot take the time to master them 
before providing for their basic needs. Obstacles to professional integration can 
have a major impact both on the status of the individual, and on the quality of the 
socio-economic tissue of the host community. 
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The ‘language for work’ approach? promoted by the European Centre for Modern 
Languages of the Council of Europe (Beacco, Krumm, Little, & Thalgott, 2017; 
Braddell & Griinhage-Monetti, 2018), proposes to develop language skills and 
professional competence at the same time. This methodology has had no adequate 
response in Italy, even if the country is dealing with stable migratory flows (Centro 
Studi e Ricerche IDOS, 2019). At the institutional level, the country lacks specific 
courses for migrants wishing to enter the workforce, an appropriate variety 
of teaching materials for beginners, and experts supporting language teachers 
(Michelini, 2020). 


Therefore, language teachers struggle to find specific teaching aids. In the 
focus group and surveys conducted for the AMIF? project La lingua italiana 
per lintegrazione e per il lavoro, all Italian L2 teachers reported difficulty in 
finding both beginner-specific teaching materials and specialised content which 
is reliable and responds to work-related needs (Cognigni, Michelini, & Vitrone, 
2022, pp. 80-93). 


With the shortage of both sector experts and an appropriate variety of teaching 
materials for beginners, the objective of this project is to enable language teachers 
to create work-related teaching content. Awaiting reorganisation from the Italian 
government training scheme aimed at adult migrants, the idea is to enable teachers 
to undertake this mode in their educational institutions, hence improving beginners’ 
likelihood of access to work. 


2. Prototype and procedure 


To identify the most strategic economic sector in Italy, statistical research was 
conducted, the chosen realm being catering (Michelini, 2020). The focus was 
placed on an A2 CEFR‘ level audience, a wide group who relies on a limited 
number of sector-specific training aids (Cognigni et al., 2022, pp. 80-93). Hence, a 
web application and an IT procedure were introduced to enable language teachers 
to collaboratively design specialised multimedia teaching content, applicable as 
the most reliable in relation to the work context. 


2. European experiences on LanguageforWork at https://languageforwork.ecml.at/ 
3. Asylum, Migration and Integration Fund. 
4. Common European Framework of Reference for Languages. 
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The IT prototype for content creation consists of different IT environments 
(Figure 1), such as a web application, a wiki? aimed for collaborative 
implementation, a storage space for multimedia teaching items, and potential 
future training. Furthermore, a sector-specific A2 level syllabus as training model 
and wiki’s interface is needed, as well as a tailor-made specialised corpus — 
RISTO — and a collection of specialised multimedia material. 


Figure 1. IT prototype structure 


Wiki/ Collection 
Specific of 
A2 multimedia 
syllabus material 


Specialised 
Corpus 
RISTO 


The wiki application was created in PBWorks®. Its interface shows the syllabus 
with its typical catering communicative acts — the necessary learning objectives 
(Figure 2). It enables teachers to implement entries of the virtual syllabus picking 
elements to be processed from a multimedia collection — which stores previously 
selected multimedia items — or from outside of it, thus enriching the collection and 
the corpus with new context-related material. Thanks to teachers’ skills, content 
can be refined through the specialised corpus, then adapted to meet learners’ 
language levels, and finally, redeposited within the digital syllabus-wiki entry, 
ready to deepen a precise learning objective. 


5. A wiki is a collaborative website which allows authors to edit and store information in a shared environment using a browser 
interface (Cantoni & Tardini, 2008, p. 31). 
6. PBworks. (https://www.pbworks.com/). 
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Figure 2. Wiki’s interface’ 


ASPETTI FUNZIONALI 
© Funzione personale 
© Esprimere intenzione e capacita di fare qualcosa 
© Esprimere le proprie necessita (per es. richiedere un giorno di permesso) 
© Esprimere opinioni personali su sapori, procedimenti di cottura etc. 
© Informare sul proprio stato di salute 
© Parlare dei propri gusti 
© Funzione interpersonale 
© Augurare e rispondere a un augurio 
© Coordinare il proprio lavoro con quello del superiore e dei pari (per es. comprendere le indicazioni di lavoro dello chef 
© Fissare un appuntamento 
© Offrire (bevande, cibi) 
© Ripetere parti del discorso altrui per confermare la comprensione (per es. chiedere conferma al cliente circa la sua ordinazione) 
e FUNZIONI REGOLATIVO-STRUMENTALI 
© Chiedere perché si esegua un procedimento 
© Comprendere testi prescrittivi legati alla professione 
© COMPRENDERE/DARE ISTRUZIONI SU FAS! Di PREPARAZIONE DI PIETANZE 
© Dare/ricevere raccomandazioni e/o consigli 
© Eseguire gli ordini ricevuti 
© Eseguire semplici procedure di primo intervento (per es. gestire un piccolo/grande ustionato, bloccare emorragie da taglio) 


Teachers can store various multimedia resources in each syllabus-wiki entry 
(Figure 3). These could be portions of written language, images, audio, files in 
various formats, external links, and videos (Figure 4), which are particularly 
valuable as a context for simulation, preparatory to real practice. 


Figure 3. Wiki’s resources® 


(Q@ Share this page 
(Gi Put this page in a diferent folder 


COMPRENDERE O DARE ISTRUZIONI SU FASI DI 4 Add Tags 
PREPARAZIONE DI PIETANZE P> Control access to ths page 
Copy this page 


last edited by & EM 16 hours ago 


Navigator 
1) Casella di testo @ Consulta la risorsa_ \_. FUNZIONE REGOLATIVO-STRUMENT... 
2) File allegati @ Consulta la risorsa_ (Ge COMPRENDERE 0 DARE ISTRUZIONI - Ese... 


3) Immagini Risorsa non disponibile © Chiedere perché si esegue un procedinento 
4) Audio Risorsa non disponibile 


5) Video @ Consulta la risorsa_ 
6) Link @ Consulta la risorsa. 


* COMPRENDERE O DARE ISTRUZIONI SU FASI DI 


* Comprendere testi prescrtivi legati ala professione 


* Dare 0 ricevere raccomandazioni o consigh 
* Eseguire gl ordiniricevuts 


~_Esequire sempiic procedure di primo intervento (ustioni, ¢} 


7. Excerpt of the wiki’s interface showing virtual training objectives to be implemented. 
8. Excerpt of multimedia resources provided in each wiki entry. 
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Figure 4. Videos 


COMPRENDERE O DARE ISTRUZIONI - 5) Video 


ed by EM 2 weeks ag 


Se sono presenti. puoi attivare i sottotitoli 


i= 
cliccando sul pulsante in basso a destra. + 


Multimedia teaching items are designed to meet basic training needs, supplementing 
traditional paper-based teaching aids. Moreover, a complete training course using 
only the application is imaginable as a stand-alone digital textbook. In this regard, 
students report a positive reception to the use of wikis for L2 learning and their 
participation in courses significantly increases (Sanchez-Gomez, Pinto-Llorente & 
Garcia-Pefialvo, 2017). 


2.1. RISTO corpus 


Tailor-made in Sketch Engine’, the specialised corpus RISTO represents the 
language of catering. BootCat'® was added for the semi-automatic retrieval of texts 
from the web. Currently, it contains about 350 texts and is continually expanding. 
Its focus is on researchers of specialised languages and their examination of 
communicative productions “in their social use” (Pietrandrea, 2012, p. 272). 
In addition, teachers can benefit from corpus resources to improve pedagogical 
practice, and when designing data-intensive teaching activities (ROmer, 2011). 


RISTO compensates for teachers’ limited specialised knowledge, enabling both 
searches and checking of specialised linguistic models, and helps teachers create 
portions of written language useful for various educational purposes. In the 
absence of a sector expert, teachers are brought closer to information accuracy and 
trustworthy introductory professional information is provided (Figure 5). 


9. Sketch Engine. (https://www.sketchengine.eu/). 
10. BootCat. (https://bootcat.dipintra.it/). 
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Figure 5. Concordance in RISTO! 


6 CONCORDANCE i seniziiristorazione < Gimme@ OS © MB 


‘simple prepara 23 (108.07 pe milion) ast @e@%X==e88 —- B kwe~r + © x 


Right 


D Details Left context KWIC 


diverdura e delle guamizioni, </s><s> Nei ristorant italiani ha una posizione di ilievo perché prepara i primi piatti.</s><s> CHEF PATISSIER - 3000/3600 euro. </s><s> Prepara dessert e pice 


i ilievo perché prepara i primi piatti. </s><s> CHEF PATISSIER - 3000/3500 euro. </s><s> Prepara dessert e piccola pasticceria. </s><s> Negli alberghi fa anche pane e cestino di sfoglie per! 


i pasti AIUTO CUOCO: Supparta il cuoco nella preparazione dei pasti: lava, taglia le verdure, prepara le insalate, compone i piatti, controlla la cottura dei cibi, si occupa inoltre della pulitura di frut 
doc#14. attraverso carretti processi di manipolazione, cottura, conservazione degli stessi. </s><s> Prepara gli ingredienti delle varie pietanze, controlla la cottura, lava le stoviglie, pulisce e riassetta lat 
doc#14_ la cucina, rifomisce di alimenti le celle frigorifere. </s><s> Aiuta nella preparazioni dei piatti, prepara antipasti, cura la cottura degli alimenti, pulisce e taglia alimenti, prepara salse. </s><s> Coz 
doc#14 sparazioni dei piatti, prepara antipasti, cura la cottura degli alimenti, pulisce e taglia alimenti, prepara salse. </s><s> Coadiuva lattivita dei cuochi in cucina, pulisce le verdure, controlla le cotture 
doc#14_ , impiatta. </s><s> Da supporto al cuoco nella preparazione dei piatti lava, taglia le verdure, prepara e cuoce i cibi. </s><s> Si occupa della preparazione delle verdure e dei cibi prima di essere 
doci#14 e realizzazione di piatti e coordina i cuochi e gli aiuto cuochi. </s><s> CUOCO PIZZAIOLO: Prepara limpasto e la base pera pizza, la quamnisce e Ia inforna controllandone la cottura. </s><s> 
doc#14limpasto e la base per la pizza, la quamnisce e la informa controllandone la cottura. </s><s> Prepara ivari condimenti della pizza, pulisce il piano o il Banco di lavoro, il figo della pizzeria e fimpa 
21 9 propedeutiche alla crescita professionale. </s><s> Ecco cosa fa un aiuto cuaco. </s><s> Prepara_gliingredienti: pesail necessario per realizzare un piatto; lava le verdure; taglia le verdure sec 
21 tti compiti banali solo in apparenza, che peré richiedono competenza ed esercizio. </s><s> Prepara_piatti semplici, che non richiedono l'attenzione del personale pid esperto. </s><s>Pulisce g 
izzati in un tipo di piatto particolare. </s><s> C’8 quello addetto ai piattidi carne, quello che prepara i piatti di pesce, il pasticcere e cosivia. </s><s> I commis di cucina sono gli assistenti dei c 
piti diversi. </s><s> Un commis pulisce il pesce, un altro pulisce le verdure, un altro ancora prepara le salse. </s><s> Di tanto in tanto cambiano partita o vengono “prestati” alla partita che hag 
siuto cuoco sono meno interessanti di quelle di un cuoco, inutile negarlo. </s><s>Un cugco prepara ipiatti e lavora al mend, di solito insieme al responsabile di sala. </s><s> L'siuto cuoco pela 
ntinua a chiamarla schiscetta (nome deltipico porta alimenti), chi invece segue la moda si prepara ogni giorno la sua lunch box. </s><s> Chiamatela come vi pare, la cosa certaé che portarsi [fi 
6 ebrigata. </s><s> Il rola in cucina Ogni commis lavora alle dipendenze di un capopartta, e prepara tutto quello che viene richiesto da questultimo o dallo chef de cuisine. </s><s> ll commis 6+ 


lozucchero e la vanillina nel fullatore; unre la farina mescolando.... </s><s> Innanzitutto si prepara il pan di spagna montando molto bene le uova con lo zucchero e la vanillin nel fullatore; siu 


OHDoOoBo0oo0ocooOoooo0o00000 


97. ele. </s><s> Le viandier De arte coquinaria Ricettario ed elenco di menu Gruo anitraLavae prepara lagu olanitra e mettila in una pentola chiusa; aggiungi acqua, sale e aneto. </s><s> Portal 


2.2. Technology, collaboration, and teacher’s skills 


A preliminary survey was carried out through close and open-ended questions 
in Google Forms during a teacher ‘language for work’ training course in digital 
environment within the AMIF project. A sample of 44 respondents of teachers 
and teachers in training provided the analysed data. Regarding the corpus, 88.6% 
of beginner-user respondents!* would use it again ‘to compensate for limited 
domain knowledge’ or simply ‘to find good examples of specialised language’. 
Asked about their main needs in creating teaching materials, the majority 
emphasised ‘finding textual content’, immediately followed by ‘finding audio- 
visual content’. 


In this collaborative project — taking its cue from constructivism (Jonassen, 1994), 
numerous teachers can contribute, intervening in the creation phase, relying on the 
expertise of other members, to the benefit of time and quality of the final product. 
In this regard, 95.5% of the sample rated the co-construction of ‘language for work’ 
teaching activities in the digital environment ‘very positively’. Finally, 97.7% 
declared their interest in repeating a course focused on the collaborative creation 
of multimedia teaching materials, thus attesting to a growing interest in the use of 
technology in design and classroom training. 


11. Excerpt of a concordance in RISTO corpus. 
12. The sample received training and tried the tool during the course. 
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3. Conclusions 


In this project, technology has been adopted to exploit the advantages of multimedia 
use in future language training. As preliminary data suggest, specialised teaching 
content design can happen in an IT collaborative environment, utilising the teaching 
teams’ competence and the impact of technology on educational design. With the 
study still ongoing, definitive results are yet to be reached and final surveys need to 
be conducted. However, preliminary results demonstrate the strength of the content 
creation method and a first digital prototype for adult migrant’s work-focused 
training has been created. Moreover, surveys confirm that IT brings significant 
advantages in strengthening teachers’ design skills, as well as facilitating the 
sharing of common knowledge and expertise. Finally, this model can easily be 
replicated for other professions and language levels. 
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Abstract. This article introduces self-regulation features with Partial and 
Synchronized Caption (PSC) for practicing listening skills and fostering self- 
paced and autonomous learning. PSC generates a partial caption by focusing 
on acoustically and lexically difficult words and synchronizes each word’s 
appearance with the speaker’s speech. The aim is to reduce textual density in the 
caption and encourage more listening than reading. Given that difficulty translates 
differently for individual learners, we created a user-friendly interface that allows 
a more individualized experience with our generated caption. We collected the 
user-configuration log data of 33 intermediate English learners to analyze learner 
behavior during self-paced practice, along with the comprehension scores of the 
subsequent tests and learner feedback. The data showed differences in learners’ 
strategies with a relative tendency to reduce their reliance on captions. Self- 
paced practice was received positively by most but not all learners, indicating the 
importance of individual differences in self-regulated listening. 


Keywords: smart captioning, L2 listening, self-regulated learning, listening 
difficulties. 


1. Introduction 


In the sphere of second language acquisition, technologies have changed the way 
we train and develop skills. However, technologies for teaching and learning 
listening skills have been limited for many years to the use of captions and 
multimedia controllers (Gass, Winke, Isbell, & Ahn, 2019). Numerous research 
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studies have been conducted on the benefits of captions in classrooms, such as 
increased comprehension and vocabulary acquisition (Fievez, Perez, Cornillie, 
& Desmet, 2020), but studies also indicate that captions could promote reading 
over listening with increasing reliance on the text (Leveridge & Yang, 2013). 
Captions also ignore the requirements of individual learners by providing full text 
to everyone, regardless of level. Therefore, there is a need to open a window onto 
the development of contemporary tools for listening and pedagogy with particular 
attention to different learners’ needs. 


In this context, self-directed learning plays a critical role. It emphasizes that 
individuals take the initiative in their learning process by diagnosing their needs, 
adapting the appropriate learning strategies, and evaluating learning outcomes 
(Knowles, 1975). In the case of listening, learners are dealing with fleeting speech 
and need to process the input quickly before the next piece of information is 
received. Putting the control of listening input in the hands of the listener will allow 
for self-paced learning, which personalizes the learning practice, supports the self- 
regulatory processes (Kitsantas & Dabbagh, 2011), fosters the sense of agency, 
motivation, and positive learning experience (McBride, 2011), reduces cognitive 
overload (Ozcelik, Van den Branden, & Van Steendam, 2019), and improves 
information processing and performance (Roussel, 2011). 


Learning from the ample research available on the L2 listening, with a particular 
focus on the individual learner’s strengths and weaknesses, we introduce PSC with 
enhanced self-regulating features that allow for generating captions for different 
language learners. We investigate how learners use self-regulated captioning and 
how it affects their performance. 


2. Self-regulation using PSC 


PSC provides for a limited number of words by introducing features that enable 
the detection of difficult words/phrases. It benefits from automatic speech 
recognition systems and corpora to identify challenging acoustic and lexical 
video segments. It further provides word-level synchronization for smoother 
audio-to-text matching. We considered a combination of acoustic and lexical 
features, ranging from acoustic neighbors, speech rate, and word frequency, to 
conduct a rigorous calculation that distinguishes easy from problematic words 
(see Mirzaei, Meshgi, & Kawahara, 2018). The system prepares captions for 
different levels — beginners, intermediates, and advanced — by adjusting the 
parameters when generating the caption. Given that some learners have a better 
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vocabulary reservoir but less tolerance in coping with fast speech or vice versa, 
we have incorporated a user-friendly interface to adjust the system’s features 
and override the default parameters. This enables increasing or decreasing the 
number of words shown for personalization and promotes autonomous and self- 
regulated learning (Figure 1). 


Figure 1. PSC with self-regulation features to adjust the caption based on 
preference 


intestine 
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3. Preliminary experiment 


Our participants were 33 intermediate English learners (520~725 TOEIC scores), 
undergraduates (19~20 years old, 15 males and 18 females, from engineering and 
humanities fields) whose native language was Japanese. The whole experiment 
was conducted online. The participants connected to the experimental sessions 
using a web browser, and the entire sessions and log data were recorded. Two pilot 
sessions were conducted to help learners familiarize themselves with the system 
and suggest parameters for their level. Before watching videos, the learners were 
tested for vocabulary size and tolerable speech rate to adjust the system’s default 
parameters accordingly. They watched a series of TED talks (15~20 minutes 
each) on various topics and answered multiple-choice comprehension questions 
after each. During the watching phase, learners could play with the system to 
generate the appropriate amount of captions or keep the suggested parameters. 
The first five minutes of each video were used to find the best setting through 
self-control, and the log data of the learner’s preferable setting was analyzed once 
the setting had been decided. We also elicited learner feedback through an open- 
ended questionnaire to complement our data with the actual learner experience. 
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4. Results and discussions 


Most participants (18) took a conservative approach by starting with a high-density 
caption, keeping it for a while, then gradually decreasing it. We observed that a 
few learners (5) took a more challenging path by starting with less than 30% of 
the text. Three learners quickly switched to a higher density caption, and only 
two continued with less than 30% text throughout the experiment. Ten participants 
leaned toward a trial-and-error approach, mostly starting with a medium-density 
caption and a slight increase or decrease over time. Two of them kept a steady 
strategy and barely changed the amount of the text. 


As Figure 2 shows, the overall trend analysis shows that, on average, the learners 
tended to choose fewer words in the caption while striving to gain an acceptable 
performance. We also observed that such a decrease did not significantly affect the 
overall comprehension, and the learners’ performances showed a relatively steady 
increase. 


Figure 2. Experimental results 
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Video #7 was notably harder than other videos in terms of topic (medical), 
vocabulary (terminologies), and speech rate. For this particular video, most learners 
considered receiving more captions to deal with the video’s difficulty; however, 
given that unknown technical terms were frequently presented in this video, the 
average performance showed a decrease. This result indicates that when the input is 
high above the learner’s level or the topic is difficult, using more text in the caption 
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will not necessarily lead to better performance. This finding is essential in that when 
presenting an input to the learner, it is necessary to consider the learner’s background 
knowledge and the difficulty level of the video relative to the learner’s level. 


The analysis of self-regulation strategies used by the learners shows that they 
were primarily successful in picking the appropriate amount of words needed to 
attain a relatively acceptable performance level. Meanwhile, for some learners, 
adapting the wrong settings led to low performance, while for others, it was hard 
to find the best setting. Even though the overall performance was not significantly 
high, data from the learners’ self-disclosure showed that most learners enjoyed 
self-regulation. They felt motivated to control their learning process, which 
may promote autonomous learning, highlighting the effectiveness of using self- 
regulated methodologies for practicing listening. However, it is crucial to consider 
that this approach may not work for all learners, especially those who find it hard 
to self-regulate (Roussel, 2011). It is anticipated that through practice, learners can 
become more familiar with the system, the types of listening challenges they face, 
and the areas they need to improve. Additionally, learner feedback revealed that 
for those overly reliant on the caption, the idea of using partial text is not easy to 
adapt. For this group, it may take significantly more time to feel confident enough 
to decrease the dependence on the caption slowly. 


5. Conclusions 


This article introduced a smart caption with adjustable parameters to meet the 
requirement of individual listeners. Several experiments were conducted with 
self-regulation listening episodes. The findings revealed a positive learning 
experience while gleaning valuable insights into differences in learner preferences 
and indicating the importance of incorporating learner differences into computer 
assisted language learning systems to provide personalized L2 listening practices. 
The current study focused on intermediate learners; however, the findings indicate a 
need to fortify the results by including different proficiency levels and backgrounds 
in future studies. 
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Abstract. Following up on Montaner-Villalba et al. (2020), this research focuses 
on written competence in the English language. The huge advance in technology 
permits teachers to use mobile applications. This quantitative design research 
investigated WordPress, in its mobile version, on written competence in English as a 
Foreign Language (EFL). One treatment group (12 students) and one control group 
(12 students) of A2 (Common European Framework of Reference for Languages) 
EFL learners at a secondary school in Valencia (Spain, N=24) participated during the 
2018-2019 academic year. While, on the one hand, learners from the treatment group 
utilized WordPress in its mobile version, on the other, learners from the control group 
used WordPress in its computer-based version. The outcomes proved that the learners 
from the treatment group significantly improved their level of written competence 
in comparison with the learners from the control group. Accordingly, this research 


recommends utilizing WordPress in its mobile version in secondary education. 


Keywords: mobile blogging, computer-based blogging, written competence, EFL, 
WordPress. 


1. Mobile-assisted language learning 


WordPress is an open-source content management system platform featured by 
plugin architecture and a template system. WordPress can also be utilized in its 
mobile version. This platform has similar options to the computer-based version. 
This research aims to analyze blogging to practice EFL written competence both in 
its computer-based and mobile versions. Some empirical research (Gonulal, 2019; 
Montaner-Villalba, 2019; Ramos, 2018; Sanchez Ambriz & Martinez Balboa, 
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2018) focused on the possibilities given by mobile devices. Ramos (2018) focused 
on WhatsApp as a tool to enhance French written competence. Results showed that 
learning was successful. However, the author recommends a longitudinal research 
to check learners’ motivation. Similarly, Sanchez Ambriz and Martinez Balboa 
(2018) explored WhatsApp in EFL to promote both oral and written competence. 
Outcomes proved that learners achieved better outcomes in the post-test. Gonulal 
(2019), in his mixed-method research, explored Instagram as a tool to enhance 
EFL vocabulary and communication skills. The findings showed that learning 
was significantly positive. Montaner-Villalba (2019) focused on Instagram as a 
tool to promote EFL written competence. In this quantitative research, the author 
concluded that learners from the treatment group improved notably at the end of the 
research. However, it could not be confirmed that there was a notable improvement 
in learners from the control group. Moreover, outcomes proved to be slightly 
higher in the control group, without being initially expected. 


Regarding mobile blogging, it must be highlighted that no empirical research on 
blogging in its mobile version has been published. In this sense, Jung Jee (2011) 
offered a short introduction to a huge variety of emerging Web 2.0 as well as 
mobile technologies which can be utilized not only in foreign language learning 
but also in second language learning. This makes this current research important in 
the field of study related to mobile-assisted language learning and, particularly, to 
EFL learning in secondary education in Spain. 


2. Method 


2.1. Participants 


Two different groups of A2 level EFL learners participated in this research. They 
were chosen randomly. The first group consisted of 12 students in the treatment 
group (m-blogging). The second group was composed of 12 learners in the control 
group (computer-based); 24 learners, aged 14-15, participated in this research. 
They were studying their fourth year of secondary education at the time of this 
experiment at a state secondary school in Valencia. 


2.2. Materials 


This study utilized a quantitative research design including a pre-/post-written task 
group design for both the treatment group and the control group. Four different 
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written tasks were administered to the learners participating in this experiment. 
The two initial written tasks were aimed at checking students’ level before the 
experiment commenced. The two post-written tasks assessed learners’ improvement 
of EFL written competence. 


2.3. Procedures 


The initial digital written task was administered at the beginning of the year, 
2018-2019, during the fourth week of September. Before commencing the 
experiment, we considered it adequate to offer learners three sessions to teach 
them how to utilize WordPress. These sessions were developed in the three weeks 
before the initial written task was given to learners. The learners’ outcomes were 
recorded for further correlation to the scores of the final written online task, 
which was given in the last week of May 2019. Table | clarifies the stages of the 
experiment. 


Table 1. Procedures 


Procedures When Description 


Initial written task Beginning First Term Initial digital written 
task takes place 


Presenting experiment Second week September Teacher introduces 
experiment, explains 
aims, and methodology 


Beginning experiment Three first weeks Three sessions are focused 
of September on explaining how 
WordPress functions 


Final written task End of the academic year Final online written 
task develops 


The outcomes of both the initial and the final online written tasks were analyzed to 
verify the research hypothesis. 


3. Outcomes 


In this section, the different outcomes of both the initial and the final written 
digital tasks are analyzed considering quantitative methods. The collecting data 
was made through WordPress in its computer-based version (control group) and 
via WordPress in its mobile version (treatment group). The means of the various 
variables were calculated through Excel. 
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3.1. Treatment group? 


Firstly, Figure | addresses the outcomes of the initial online written task; secondly, 
Figure 2 shows the average of the post-writing. 


Figure 1. Average outcomes of the initial writing digital task, treatment group 
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Figure 2. Average outcomes of the post-writing online task, treatment group 


3.2. Control group 


Next, the quantitative results of the control group are analyzed. Figure 3 addresses 
the results of the initial digital written task and Figure 4 shows the average of the 
post-writing. 


2. The meaning of the numbers in Figures 1, 2, 3, and 4 are related to the grades which are used within the Spanish educational 
system. The grades or marks from 0 to 4 imply failure. Next, the grade of 5 means ‘Pass’, the mark of 6 implies ‘Good’, the 
grades of 7 and 8 mean ‘Very good’ and, finally, the grades of 9 and 10 imply ‘Very excellent’. 
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Figure 3. Average outcomes of initial writing online task, control group 
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Figure 4. Average outcomes post-writing digital task, control group 
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4. Discussions and conclusion 


Firstly, the outcomes of the initial digital written task are discussed from a 
comparative perspective between the treatment group and the control group. This 
task aimed at measuring the initial level of EFL written production by learners. The 
total average of the treatment group is 8.33 while the total average of the control 
group is 9.68. This does not fulfill the initial expectation. When comparing both 
the initial online written task (8.33) and the final one (8.92) in the treatment group, 
there is a slight improvement between them, implying, thus, that treatment group 
learners slightly improved their level of written production in WordPress in its 
mobile version. 


283 


Salvador Montaner-Villalba 


Secondly, the results of the control group learners are justified by explaining the 
differences between the treatment and control group regarding the initial online 
written task. Concerning vocabulary, the control group (9.8) is slightly higher than 
the treatment group (8.35). In this task, both groups were required to write about 
their daily actions. Regarding grammar, the control group (9.79) is significantly 
much higher than the treatment group (7.82). The control group learners possibly 
scored better results because they correctly identified grammatical issues required 
in this task. It is curious to observe here that the control group shows better results 
than the treatment group. This was not initially expected before commencing the 
experiment. 


As for the organization, the control group (8.73) is notably higher than the 
treatment group (6.09). This huge difference between both groups can possibly 
be explained by the fact that control group learners were more aware than the 
treatment group of the relevance of writing cohesive and coherent texts and using 
well-structured paragraphs; also taking into consideration that writing in computer- 
based WordPress is easier than in mobile version. This was not initially expected. 


Thirdly, the outcomes of the final online written task are discussed by comparing 
the initial and the final tasks. The content is notably higher in the treatment group 
(8.7) than in the control group (5.1). In the final task, learners wrote about their 
experience in this experiment. It is obvious that the outcomes of the treatment 
group are better since they understood what was required. As for the organization, 
the results of the treatment group (9.05) are higher than the control group (6.25). 
Learners from the treatment group managed to write well-structured texts 
satisfactorily while learners from the control group did not. 


In relation to grammar, the results of the treatment group (8.5) are better than the 
control group (5.9). Control group learners confused the conditional form would + 
infinitive and, instead, they utilized the future form wi// + infinitive. The complete 
lack of connectors was key in their low marks. As for vocabulary, the treatment 
group (8.7) is higher than the control group (6.7). Learners from the treatment 
group utilized more varied and richer vocabulary than learners from the control 


group. 


Related to spelling, the treatment group (9.6) is higher than the control group 
(6.2). While treatment group learners paid more attention to spelling, learners from 
the control group did not obtain good marks in the final written digital task. The 
outcomes proved that the treatment group improved slightly whereas the control 
group decreased significantly. As a whole, the total average of the treatment group 
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is notably higher than the control group. These results were initially expected 
before commencing the experiment. 


From the outcomes given through the tool Excel, we can clearly state that the 
results improved significantly at the end of the experiment in the treatment group in 
comparison with the control group. This means that the participants in the treatment 
group improved their level of EFL written competence through WordPress in its 
mobile version while the outcomes from the control group in the final written online 
task decreased notably if we compare them with the results from the initial digital 
written task. For this reason, it is recommended that learners practice EFL written 
competence through WordPress and, particularly, in its computer-based version. 
Moreover, we would encourage scholars to do further research on m-blogging 
related to enhancing other linguistic skills. 
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Abstract. There is no doubt that telecollaboration currently plays an important role 
in foreign language learning and, not less so, in the field of English for Specific 
Purposes (ESP). Guth (2020) highlighted how telecollaboration has rapidly evolved 
in the past years as an innovative approach, and how it has brought together a whole 
community of academics and researchers interested in the field. In this paper, a 
brief overview of the various presentations that took place in the EuroCALL 
Computer-Mediated Communication (CMC) Special Interest Group (SIG) 
Symposium is offered. The paper introduces four projects based on telecollaboration 
contextualised within an ESP classroom in higher education. The first one focuses 
on a collaborative debate project using English as a lingua franca; the second 
elaborates on improving learners’ pragmatic skills through telecollaborative role- 
plays; the third describes an immersive Virtual Exchange (VE) aiming to foster the 
students’ civic and entrepreneurial competence, while enhancing their intercultural 
communicative competence. The fourth project aimed at improving students’ 
business communication and management skills in English in a multicultural 


environment. 


Keywords: telecollaboration, virtual exchange, languages for specific purposes, 
English for specific purposes, pragmatics, virtual reality, intercultural competence. 
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“7 Introduction 


CMC can be described as any human communication which takes place with the 
aid of two or more electronic devices. The term CMC has traditionally referred 
to diverse kinds of communication, such as e-mails, e-forums, instant messaging, 
etc. This term has also been applied to other forms of text-based interaction, 
such as text messaging (Thurlow, Lengel, & Tomic, 2004). Significant research 
on CMC is mainly focused on the social effects of diverse computer-supported 
communication technologies. Telecollaboration or VE is indeed one of the most 
enriching and interesting forms of authentic communication in foreign language 
learning in general and in ESP settings, in particular. This is what triggered the 
EuroCALL CMC SIG to bring together six experts to explain current practices in 
telecollaboration which can enhance language learners’ motivation. 


2. Proposals of the symposium 


2.1. The what, why, and how of telecollaboration 
projects in ESP in higher education 


Inherpresentation, Gimeno-Sanz explored two projects combining telecollaboration 
and ESP in higher education. Both projects were based on collaboration and product 
creation tasks as described by Harris (2002) and Helm and Guth (2010), which are 
the most demanding and, therefore, the least often used. The first project between 
students from two Spanish universities using English as a lingua franca was based 
on a collaborative debate project. The post-project questionnaire revealed that 
learners perceived they had primarily improved their speaking and listening skills, 
as well as acquiring new vocabulary relating to their field of study. In addition, 
students pointed out they had improved a number of ‘life skills’ such as public 
speaking, critical thinking, articulating thoughts, learning to think on their feet, 
controlling emotions when speaking in public, and improving their presentation 
skills. These results were aligned with their expectations, as evidenced by a pre- 
project questionnaire. 


The second project, with Spanish ESP students and US students of Spanish as 
a foreign language, had the clear goal of developing the students’ intercultural 
awareness. It was conducted bilingually through synchronous and asynchronous 
tools to give both sets of learners equal opportunities to practise their foreign 
language. The pre-project survey showed that, overall, the students were open- 


287 


Salvador Montaner-Villalba et al. 


minded about discussing intercultural issues and very positive toward collaborating 
with fellow students from different cultural backgrounds (Gimeno, 2018). They 
were happy to use information and communications technologies for language 
learning purposes and did not think that communicating in writing through a private 
social network or orally through a video-conferencing system hindered authentic 
intercultural communication and interaction. The surveys also showed that the 
learners’ expectations correlated with their stated post-project gains. Regarding 
specific benefits, there was clear evidence that the Spanish students saw the project 
as being instrumental for their English as a foreign language improvement, which 
was one of the project aims. The artefact students had to produce collaboratively in 
groups of four (two US-based and two-Spain-based) was a bilingual radio podcast 
simulating an interview where each cohort summarised their findings in relation to 
the topics under discussion. 


2.2. Does participating in a telecollaborative project foster 
the acquisition of apologies? Insights from the ESP context 


Since Yus (2011) coined the term cyberpragmatics, which is “the application 
of pragmatics to online interactions” (Orsini-Jones & Lee, 2018, p. 26), the 
interest in analysing the pragmatic implications of online encounters has evolved, 
although the field of interlanguage pragmatics in computer-assisted language 
learning is still under-investigated (Sykes & Gonzalez-Lloret, 2020). Despite 
this, Orsini-Jones and Lee (2018) argue that cyberpragmatics is of paramount 
importance in telecollaboration. Based on these presumptions, Di Sarno-Garcia 
presented a comparison between the results obtained from a control group (n=17) 
and an experimental group (n=7) that participated in a telecollaboration project. 
Di Sarno-Garcia aimed to demonstrate that telecollaborative encounters are a 
suitable environment for the development of L2 pragmatic competence and for the 
acquisition of the speech act of apologies, which is usually a problematic speech 
act that they are likely to perform in their future professional life. Participants 
from both groups were aerospace engineering students from the Universitat 
Politécnica de Valencia (Spain), and they performed six open role-plays to elicit 
their use of apologies. The difference lay in the fact that participants in the control 
group carried out the task in pairs with their Spanish classmates in a face-to-face 
setting, while participants in the experimental group conducted the task with L1 
or highly proficient speakers of English from the University of Bath (UK) through 
synchronous Zoom sessions. Their apologies were coded following a taxonomy 
based on Blum-Kulka and Olshtain (1984), Leech (2014), and Martinez-Flor 
(2016). The results of the qualitative analysis revealed that the apologies produced 
by the experimental group were more complex and sophisticated as they were 
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performed using a wider variety of strategies compared to those of the control 
group. On the other hand, findings from the quantitative analysis carried out 
through an Eta coefficient revealed a significant correlation (r=.71) between the 
number of strategies used and the modality where they were performed. The 
descriptive analysis of the responses obtained in the pre- and post-test of each 
group demonstrates that there is a higher tendency of improvement in the case of 
the experimental group, while in the case of the control group the means reveal a 
regression in the responses obtained to different questions. Therefore, we can claim 
that telecollaboration projects are an adequate environment for the practice and 
acquisition of the speech act of apologies within an ESP context. 


2.3. Sustainable development goals meet immersive VE: 
the youth entrepreneurship for society in a 3D initiative 


Immersive telecollaboration, immersive VE, or 3D VE involves the interaction, 
communication, and collaboration of geographically dispersed groups of learners 
via immersive virtual reality by means of engaging participants in 3D-embodied 
interaction. To our knowledge, there is very limited research exploring 3DVE for 
language learning in ESP contexts. To address this gap, the 3DVE research project 
YES3D was launched with the participation of 16 business English students from 
the Cyprus University of Technology and 26 students from the Universitat de 
Valéncia. Through the completion of different social entrepreneurship project tasks, 
participants produced various collaborative digital artefacts in their plurinational 
teams. Interaction, communication, and collaboration were mediated by various 
tools, including high immersion virtual reality technologies. The co-authored 
digital artefacts were analysed in terms of their alignment with the sustainable 
development goals proposed by UNESCO (2017a, 2017b), thus shedding light on 
how YES3D contributed towards fostering the students’ civic and entrepreneurial 
competence while enhancing their intercultural communicative competence within 
a discipline-specific, ESP curriculum. This was achieved through the creation of 
highly immersive experiences in the participants’ synchronous interactions which 
afforded opportunities for increased engagement and effective collaborative work 
on a meaningful common goal. 


2.4. VE in the business communication classroom: 
catalyst for employability skills and intercultural 
competence development 


This presentation (Koris and Vuylsteke) described the benefits of VE on students’ 
knowledge acquisition and skills development. The VE project described was 
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implemented in the field of English business communication by a Hungarian 
and a Belgian university for three consecutive academic years between 2019 
and 2021. Students participating in the VE represented a large variety of 
nationalities with diverse cultural and educational backgrounds. During the 
collaboration, participants worked online in virtual teams simulating real business 
communication practices in English (Swartz & Luck, 2018). Capitalising on the 
multicultural diversity of the teams, students engaged in discussions about the 
cross-cultural differences in their local business environments, job markets, 
recruitment processes, job applications, and interviews. At the end of the project, 
students participated in a real online job interview with professional recruiters. 
As a final assignment, students presented a digital portfolio on their learning 
experience, professional development, acquired employability skills, and 
intercultural competence. The qualitative analyses of student team reports, digital 
portfolios, and results of pre/post-project surveys of the three iterations revealed 
that students developed employability skills by facing new challenges in the 
culturally diverse business world. Not only did it allow students to improve their 
business communication and management skills in English in a multicultural 
environment, but it also challenged their intercultural communication competence 
and global collaboration skills. Hence, VE is an efficient means of and a catalyst 
for developing students’ intercultural competence without physically leaving 
their classrooms. 


3. Conclusion 


The EuroCALL CMC SIG symposium participants provided ample evidence 
demonstrating that telecollaboration or VE practices in a languages for specific 
purposes context has more advantages than drawbacks. Through these projects, 
students can improve their linguistic skills and gain intercultural competence, two 
essential assets in today’s global world that would otherwise not be possible in a 
traditional classroom setting where students only interact with fellow students or 
with their teacher. It was also demonstrated that pragmatic skills, which are crucial 
in communication, can also be developed through telecollaborative activities. The 
authors highlighted several requirements for these projects to be successful, that is, 
strict coordination between the project tutors from both participating universities, 
clear instructions provided to the learners at the start of the project, implementation 
of questionnaires to gather both expectations at the outset and satisfaction upon 
conclusion, a clear artefact to be produced collaboratively by the mixed cohorts of 
students, full integration into the course programme, clear evaluation guidelines, 
and accounting for student grading. 
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Abstract. While previous studies have established the significance of CALL 
(Computer-Assisted Language Learning) in language education, the extent to which 
CALL influences language teachers’ well-being and their professional development 
has not yet been fully understood. This echoes Dérnyei’s (2018) argument that 
although language teachers are the most important factors in learners’ success, their 
contributions have not been considered sufficiently. As such, this short paper presents 
an ongoing mixed-methods project investigating Vietnamese and Japanese language 
teachers’ well-being and the role of CALL in the ecology of language teaching at 
both personal and work levels using Bronfenbrenner’s (1979) ecological system 
framework. The primary method is the grounded theory analysis and preliminary 
results from Vietnamese teachers’ semi-structured interviews which have shown the 
importance of CALL in language teachers’ well-being’s ecology. Also, the role of 
the community of practice has also emerged as an informal support in language 
teachers’ ecological well-being system. 


Keywords: language teachers, CALL, well-being, ecological perspective, 
professional development. 


1. Introduction 


Teaching English is not an easy endeavor, especially when teachers have 
to handle multiple tasks, some of which include administrative paperwork, 
teaching methodologies, and classroom management techniques. Language 
teachers inevitably need to find ways to maintain their physical and mental 
well-being on a daily basis. Mercer (2021) argues that problems related to well- 
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being in education, in English Language Teaching (ELT) particularly, should be 
solved as soon as possible when more and more mental health issues are being 
reported, especially depression and anxiety. Some uncovered by previous studies 
are increased emotional labor, shifting identities, energy-intensive teaching 
methodologies, language anxiety, intellectual demands (Benesch, 2017), and 
more importantly, emotional exhaustion (Daniels & Strauss, 2010). CALL is no 
longer an uncommon approach in a language classroom. Research investigating 
CALL and its affordances has been the focus of many scholars. Nevertheless, 
studies investigating the importance of CALL to language teacher well-being 
are still scarce. For this reason, as part of a PhD project investigating language 
teacher well-being in Vietnam and Japan from the ecological perspective, this 
paper sets out to study the role that CALL plays in each layer of language 
teachers’ ecological well-being system and the extent to which CALL affects 
language teachers’ well-being and their professional development, particularly 
their physical and mental health. 


2. Literature review 


2.1. Language teacher well-being from the ecological perspective 


To understand language teachers’ well-being, it is important to perceive them as 
individuals whose well-being is unique and paramount to the whole ecological 
system of ELT. In this respect, Bronfenbrenner’s (1979) ecological system 
framework may be helpful. The ecological system of a person needs to be taken 
into consideration in order to understand their development (Bronfenbrenner, 
1979). He proposes that the system comprises four main subsystems, namely 
the microsystem, mesosystem, exosystem, and macrosystem. According to Cross 
and Hong (2012), the microsystem is an environment in which teachers, students, 
and parents communicate and influence one another in a multidirectional and 
reciprocal relationship. The mesosystem is the place where teachers do not 
participate directly, yet they are affected on a daily basis. According to Price and 
McCallum (2015), it includes interactions among different microsystems such as 
teachers’ work and their private life. The macrosystem is the biggest environment 
which encompasses the entire ecological system of language teacher well-being. 
Some typical factors are politics, economy, legal, and cultural influences. Even 
though teachers do not participate directly, they are influenced by this system 
because they need to follow the norms and understand what is happening in their 
working culture. 
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2.2. | CALL as a participant in the ecology 
of language teacher well-being 


Thanks to previous research into technology, the assistance CALL provides 
language teachers with can be categorized into two main parts, that is, acceptance 
and persistence toward CALL. If positive attitudes toward CALL result in a higher 
frequency of CALL application and enhance perseverance upon encountering 
CALL-related difficulties (Drossel, Eickelmann, & Gerick, 2016), insufficient 
training and preconceived ideas toward CALL (Stockwell, 2012) might lead to 
CALL rejection. Hence, CALL is not only a teaching approach, it also plays a 
key role in teachers’ professional development and well-being. Helping teachers 
handle CALL-related difficulties and put it into practice might result in a better 
outlook toward the profession and motivation for long-term pursuit. Therefore, 
this project focuses on studying CALL and the role it has in each system of the 
language teacher well-being’s ecology. Three research questions encompassing the 
inquiry are as follows. 


¢ To what extent does CALL influence the ELT ecology of language teacher 
well-being? 


¢ To what extent does CALL influence language teachers’ professional 
development from the ecological perspective? 


¢ To what extent do language teachers in Japan and Vietnam differ in their 
perspectives on the role of CALL in the ecology of their well-being and 
professional development? 


3. Methodology 


For this PhD project, participants are teachers at tertiary level from Japan and 
Vietnam. A mixed-methods design is employed using the grounded theory approach 
and data triangulation as the primary method. Data includes (1) semi-structured 
interviews conducted individually with 30 language teachers who are currently 
working at university, (2) focus group interviews, and (3) a quantitative-oriented 
questionnaire. 


However, as this paper sets out to understand the role of CALL in the ecological 
system of language teachers’ well-being, only data collected from the individual 
semi-structured interviews are utilized. Three participants, two of whom have 


294 


The role of CALL in the ecology of language teachers’ well-being... 


a master’s degree, have been interviewed. They are all working in the private 
sector with at least five-year teaching experience. The interview time varies from 
45 minutes to 60 minutes focusing on three main parts and the protocol was adapted 
from Dérnyei’s (2007) advice on conducting interviews. The first part focused 
on the participants’ backgrounds and their teaching experience. The second part 
includes open-ended questions investigating the role of CALL in their well-being 
and professional development, such as ‘how do you feel about technology in 
teaching’, ‘what do you do when you encounter a technological problem’, and ‘is 
using technology stressful to you”? The interviews ended with a feedback session 
where the participants asked questions related to the interview. Interview data were 
then analyzed qualitatively using the grounded theory approach and the results are 
presented in the next section. 


4. Preliminary results 


Findings from three interviews indicate that not only does CALL play an important 
part in the microsystem (i.e. the individual level), but it also exists in other systems, 
namely meso-, exo-, and macrosystem of language teachers’ ecological well-being. 
The participants mentioned that using CALL was not for their own benefits anymore 
(the mesosystem). They need the competence to help engage their students (the 
microsystem) and increase the productivity for their schools (e.g. delivering online 
classes, handling administrative tasks, etc.). Also, the participants mentioned that 
frustrating as CALL learning may be, it was also a catalyst to initiate more informal 
conversations with their peers, which eventually strengthened the bonding in the 
workplace. 


5. Conclusion 


Overall, analysis from the interviews indicates that CALL does play a key role 
in the ecology of teachers’ well-being, especially the way it influences language 
teachers’ professional development in this ecology. This echoes with Hodstadler 
et al.’s (2020) arguments on the stakeholders teachers have to deal with in their 
profession, such as school policies or colleagues. However, CALL’s existence does 
not entirely entail complete negativity. The community of practice established 
through the CALL frustration in this study helped teachers overcome CALL- 
related obstacles and enhanced their well-being. For institutions and policymakers 
in both public and private sectors, this hopefully can become a helpful reference 
to construct a more comprehensive support system in the workplace environment 
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to help teachers and learners gain optimum outcomes. More importantly, the 
study also hopes to shed more light on how to provide teachers with trustworthy 
guidelines to help them deal with mental health issues on their own and advance 
their professions. 
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Abstract. An Corpas Cliste (‘Clever Corpus’) 1s an Irish language learner corpus. 
The corpus data comes from a purpose-built intelligent Computer Assisted 
Language Learning (iCALL) platform called An Scéalai (‘the Storyteller’) and 
comprises both audio and text, produced by second and third level learners of 
Irish. Metadata (e.g. L1, level of Irish, dialect preference, age) is saved with every 
learner account, along with data on platform engagement (e.g. speech/language 
technologies employed, time spent on task). This paper illustrates how An Corpas 
Cliste is structured and is being prepared for analysis and the methodologies and 
resources that are being used to exploit it with a view to enhancing the learning 


experience. 


Keywords: learner corpus, iCALL, Irish, speech technologies. 


1. Introduction? 


An Corpas Cliste (‘Clever Corpus’) is a learner corpus that is being collected 
within the framework of an Irish (Gaeilge) iCALL platform, An Scéalai (‘the 
Storyteller’). Two types of learner production data are being harvested within 
An Scéalai — audio and text. The current discussion centres on learners’ written 
language, which currently consists of 44,093 learner stories — a total of 5,732,397 
words at an average of 130 words per story (July 2022). Analysis of the spoken 
language will follow at a future date. 
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An Corpas Cliste represents a novel approach to the design of language learning 
technologies for Irish. The learner corpus being collected is intended (1) to serve 
as a sound empirical basis for linguistic research into the process and stages of 
Irish language acquisition (e.g. investigating the most common errors made by 
learners at various stages of proficiency), and (2) to enhance the An Scéalai iCALL 
platform itself, making it more effective for future users. Analytic results from An 
Corpas Cliste are intended to be the driver of the personalised corrective feedback, 
which is at the heart of An Scéalai. 


To appreciate the nature of the corpus and the ways in which it will be deployed, 
the background of An Scéalai is explained. 


2. Background: An Scéalai 


The An Scéalai iCALL platform is freely available as a desktop web application 
(see https://abair.ie/scealai). It has been designed from the ground up and is aimed 
at the individual language learner; 4,896 account registrations are logged, with 
circa 2,736 ‘active’ users. It is part of the ABAJR initiative, which is developing 
speech technology for Irish (particularly Text-To-Speech (TTS) and Automatic 
Speech Recognition (ASR) systems — see https://abair.ie). 


In the platform, learners write content, such as reflective diaries, stories, and essays, 
and are given feedback via (1) the latest versions of the ABAIR technologies, 
(2) grammar checker/dictionaries, and (3) teacher feedback (see Ni Chiarain 
et al., 2022). The platform provides exposure to native speaker models of the 
language (critical in the Irish context), allowing proof listening and correction 
by choosing a preferred voice from three main (very different) dialects. Another 
key technology being employed, which learners report as extremely useful, is the 
grammar checker An Gramadoir (Scannell, 2013). Dictionaries (see teanglann. 
ie) are integrated and, when used in a class setting, learners can obtain feedback 
from their teacher via written/voice notes. Learners can record themselves and 
compare recording to native speaker (synthetic) models. By integrating the 
technologies, the multiple skills of writing, reading, listening, and speaking are 
promoted in tandem. 


A major part of the rationale for building An Scéalai was to have our own 
configurable tool to conduct Second Language Acquisition (SLA) research on 
Irish, to enable the harvesting, analysis, and exploitation of the types of learner 
data the platform is designed to yield. 
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c 5 An Corpas Cliste 


Rationale: our aim is to use An Corpas Cliste as a mechanism for tracking how 
Irish language learning happens over time. By using the corpus to investigate 
learning gains over a long-term at both macro and micro levels (i.e. both the broad 
collection of language learners and individual language learning journeys), we 
hope not only to feed this information back into the iCALL platform to benefit our 
learners but also to inform curriculum development on a wider scale. 


To achieve this, we are capturing an ordered series of events by language learners. 
In the current setup, learners’ stories are saved after each edit made (if the user 
stops typing for 1.5 seconds, their story is saved). This yields large amounts of 
very detailed data. This data granularity presents challenges from an analytic point 
of view but will be extremely useful, e.g. to develop pedagogical strategies to 
deliver appropriate content to learners. By also saving learner engagement we gain 
insights into learners’ usage of individual speech and language technologies (TTS/ 
grammar checkers/ASR systems). 


The corpus data will in future lend itself well to machine learning methods, as we 
identify patterns in learning behaviour both at individual and group level. However, 
as illustrated below, extracting meaningful information from the corpus is crucial 
and we are not yet at the stage that we can depend on automatic methods to give 
reliable information. 


4. Content management 


The open-source NoSQL database management system MongoDB is being used 
to manage the An Corpas Cliste content. It is being used not only to store learners’ 
final stories and accompanying feedback (both text and audio), but also to store 
learners’ story composition journeys, including running TTS, listening back, 
performing grammar checks, receiving teacher feedback, and dictionary lookups. 
Timestamps are saved with each engagement as well as metadata provided by the 
learners. 


To date, a considerable amount of effort has been devoted to the processing of data. 
Stored in several collections of a MongoDB database, the data has been processed 
with Python to produce several categorical DataFrames. These DataFrames serve 
as an attempt to give researchers a full overview of the learner corpus. The current 
structure allows us to explore the data and answer some initial research questions. 
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Data processing is an ongoing exercise. Restructuring will be an iterative process, 
depending on the data needed to be extracted (e.g. specific age groups/time frames/ 
levels of Irish) to give insights on learner engagement/language acquisition 
processes. Ongoing collection of data is being used to calculate relevant statistics, 
(e.g. numbers of learners/stories), as well as measures of SLA progress, (e.g. 
changes in grammar/writing errors over time). These statistics, along with the 
regular manual analyses conducted by linguistic researchers serve as a valuable 
resource for creating learner models in the future, a major goal of the current 
initiative. 


5. Corpus analysis — preliminary findings 


A preliminary analysis was conducted on the entire corpus to investigate the 
most common errors being made by learners. Two different approaches were 
employed — an automatic analysis using An Gramadoir and a manual one, where 
a linguist identified the most frequent types of errors by examining the written 
text. 


The results of the first (automatic analysis) approach yielded the following as the 
‘Top 5’ error categories, (terms as encoded in the grammar checking software): 


* incorrect; 

* unknown word; 

* non-standard form; 

¢ lenition missing; and 

* possibly a foreign word. 


Unfortunately, the description of the errors detected is often not helpful to the 
learner — the most flagged error is ‘incorrect’, which is an umbrella for many 
different items where the spelling deviates from the expected. These reflect a 
variety of grammatical errors, as well as simple typos, etc. 


The result of the second (manual analysis) approach indicates that the ‘Top 5’ error 
categories involve issues with: 


* genitive case; 

¢ the lenition associated with the possessive adjective ‘a’; 
¢ relative clause construction; 

* missing acute accents (indicate long vowels); and 
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¢ misspellings associated with broad/slender vowel agreement 
(pronunciation-related). 


Although the manual analysis is preliminary and not yet the result of a rigorous 
scientific analysis, it is nonetheless clear that the two approaches yield completely 
different results from one another (note: a thorough error analysis of a corpus 
subset (n=172 learners) is ongoing). 


The take home point is that the grammar checker, as currently configured, is not fit 
for the present purpose. It simply does not differentiate adequately among different 
kinds of errors that show up in learners’ written forms. In this way it does not align 
with what the teacher would want to focus on in terms of individual grammatical 
learning targets. Furthermore, the An Gramadoir output, as with most generic 
grammar checkers, is not in any case designed for the learner. The type of grammar 
checker needed for an L2 learner would ideally be designed to provide the type of 
personalised feedback that is user-friendly for learners at different stages. 


6. Conclusions 


Current developments for An Corpas Cliste include the building of independent 
grammar checking modules for specific aspects of the language. Analysis of the 
spoken language corpora will require a different set of analytic skills. For all these 
efforts, the training of language experts is crucial. 


As mentioned above, we aim to deploy state-of-the-art machine learning 
approaches to deliver personalised corrective feedback via An Scéalai. However, 
given the frequent assumption that machine learning approaches can of themselves 
do everything, we advise caution as we are reminded that the ‘i’? in iCALL will 
continuously demand human linguistic and pedagogical knowledge in the loop. 
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Abstract. This paper explores the utilisation of Twitter as a dynamic language 
learning platform in order to create an authentic domain of use for learners of Irish 
as an Additional Language (IAL) in a primary school setting in Ireland wherein 
20 fourth class children and their parents participated. The Irish language context is 
firstly introduced, which contextualises a rationale for utilising Twitter for language 
learning. Teacher, child, and parent engagement with Twitter is then outlined with 
reference to the action research methodology employed. The repository of class 
tweets, periodic participant advisory group discussions and questionnaires, coupled 
with teacher-researcher reflection informed a qualitative data analysis. Results 
indicate notable student engagement with and enjoyment of tweeting in class as 
Gaeilge [in Irish] whereby the class Twitter account emerged as a practicable 
language learning platform. Its affordances enabled the student composition and 
publishing of short texts (tweets) and the provision of authentic Irish tweets for 
class reading. Parent and child engagement out of class, while relatively low, also 
demonstrated a promising additional domain of use, subject to the negotiation of 
orientation-related and implementation-related challenges for learners in a blended 
setting. In conclusion, recommendations to support the use of Twitter in endangered 
language school contexts are suggested. 
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1. Introduction 


The Irish language is spoken by a minority of the population on a daily basis 
outside of education (1.7%) and is categorised as an endangered language. 
While the Irish immersion education (IME) sector has experienced growth and 
success, most children in Ireland learn IAL at English-medium schools where 
Irish is a compulsory and core subject. There are ongoing concerns regarding 
children’s attainment in Irish at primary school level in English-medium 
education coupled with a lack of opportunity for learners to experience Irish 
as a living language in additional domains other than school (Harris, 2008; 
Inspectorate, 2022). 


The potential of social media to open new digital spaces for learners and to extend 
language learning opportunities is well documented. A number of critical reviews 
and studies (Barrot, 2021; Hattem & Lomicka, 2016) identify key affordances of 
Twitter for language learners such as brevity of written tasks, interaction with 
(native) speakers, learner agency, and community building. 


Digital literacies today such as social media engagement are characterised as 
participative, multifarious, and everyday (Reinhardt & Thorne, 2019) compared 
with more traditional digital activities involving bidirectional teacher and student 
exchange. Educators are encouraged to “consider involving broader audiences 
and purposes that would help students to engage with the world outside of 
the classroom” (Reinhardt & Thorne, 2019, p. 218), which resonates with the 
minority language context whereby social media can be harnessed to create 
language learning opportunities (Cunliffe, 2021). 


2. Class Twitter account engagement 


The exploration of utilising Twitter as a dynamic platform for Irish language 
learning forms part of a larger Irish language action research study informed by a 
socio-cultural theory theoretical framework, which explores Web 2.0 technology- 
mediated language and tutoring (peer tutoring and student-parent tutoring). 


A fourth class (n=20, ten years of age approximately, ten girls and ten boys) and 
their parents (n=20) voluntarily took part in the school-based study as active 
participants in partnership with the class teacher in the role of teacher-researcher. 
Parents’ consent and children’s assent were negotiated at the outset as part of a 
university-approved ethics plan inclusive of secure data retention and child- 
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friendly internet engagement stipulations. Three Action Cycles’ took place over 
the course of an academic school year. The class Twitter account was introduced at 
the halfway point of Action Cycle 1, after peer tutoring had commenced. 


A dedicated class Twitter account was set up independently of the school’s 
communication channels and it was curated by the teacher-researcher on behalf 
of the class. The modelling and co-writing model of use sought to create regular, 
short, achievable and everyday (Reinhardt & Thorne, 2019) Irish language 
writing opportunities for the class in addition to reading authentic Irish tweets 
and related potential community building (Cunliffe, 2021). Parents were invited 
to join Twitter, follow the class account and engage with class tweets with their 
child as a shared Irish language activity. A class newsletter of tweets was also 
collated and shared periodically. 


Data collection comprised of the repository of class tweets, participant advisory 
group discussions and questionnaires, and the teacher-researcher’s reflective diary 
over the course of three Action Cycles. 


Analysis of the class tweets focused on tweet authorship and subject content. 
A qualitative analysis approach was undertaken whereby a thematic analysis of 
advisory group discussions was triangulated with questionnaire and diary findings, 
and tweet analysis. 


3. Results and discussion 


Over the course of the language project, the class Twitter account produced 
459 original tweets. Figure | illustrates the distribution of tweets and original tweets 
as well as the categorisation of the content of co-written tweets. The scaffolded 
approach to tweet writing is broadly reflected in the fact that 49.7% of original 
tweets were co-written by the children and teacher. Of the co-written tweets, 60% 
reflected the learning in Irish lessons. 


3. An Action Cycle refers to sequential informed activity carried out during a designated time characterised by the identification 
and/or observation of a concern, planning and implementation, review of actions and progression to a further action cycle or 
conclusion of project. 
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Figure 1. Breakdown of class tweets 
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An analysis of children’s questionnaires and discussions indicate that a clear 
majority of the class (17/18) reported enjoyment in relation to writing and 
reading and agreed (18/18) that Twitter was a useful tool to learn and improve 
one’s Irish. Two children shared awareness of the potential to connect with other 
Irish speakers: “I think it’s really cool to see how other people and not just in 
school, they are actually using Irish and, like, what they have to say in Irish” 
(Child 4). Five children recommended that more time be afforded to reading 
Irish tweets in class. Of the ten children who expressed a desire to be involved 
with home-based Twitter activity, one child commented: “Yeah, I never got the 
chance to do it at home [reading tweets] ...But I would want to do it at home” 
(Child 17). 


In terms of parental engagement, 5/20 parents joined Twitter and followed the 
class account and three parents and their respective children engaged with the class 
account. Parents expressed their support for the role of technology, citing how the 
children were adept at its use. Unfamiliarity with Twitter was notable, in addition 
to confusion with two platforms (Twitter and Class Blog) being in use: “Well you 
see I didn’t know we could go on it” (Parent 1). Nevertheless, children reported 
notable parental engagement with the printed class newsletter of tweets. 


4. Conclusion 


The findings suggest that the children were positively disposed towards the use of 
Twitter for Irish language purposes. Parents’ feedback indicates that family use of 
Twitter outside the classroom may have been affected by unfamiliarity with the 
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application and potential intervention overload. The importance of the role of the 
teacher as curator and facilitator also emerged. 


While small-scale and context-specific, this exploratory study gives an insight 
to learner engagement with and experience of a social networking platform, 
the potential affordances of Twitter in this learning context, and the role of the 
teacher in engaging learners. The utilisation of Twitter as a scaffolded writing tool 
facilitated an agentic approach to Irish language writing, which enabled learners to 
engage with the language in a new and authentic domain of use (Cunliffe, 2021). 
The study addresses a research gap both in terms of exploring the use of social 
networking sites such as Twitter for language learning at primary level and in terms 
of the potential of such a platform in an endangered language context. 


Recommendations for future implementation based on the findings are as follows. 


¢ Utilisation of and further exploration of Twitter as a writing tool in a 
scaffolded capacity to engage primary school Irish language learners in 
realisable writing tasks. 


¢ Introduction of Twitter to parents on a phased basis whereby sufficient 
time is given to explore account set up and application affordances. 


¢ Invitation to parents to join Twitter at the latter stages of a blended 
language learning project when a greater connection with the overarching 
project aims has been established. 


¢ Facilitation of teachers to avail of the school Twitter account for the 
potential piloting of language learning prior to potential establishment of 
a class Twitter account. 
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A. 


Since the adoption of neural networks, e.g. Google in 2016 and DeepL in 2017, 
MT has developed dramatically with output becoming increasingly accurate. 
This has brought MT into a more central position in L2 education, partly 
due to its learning affordances (Lee, 2021) and partly because of its real and 


Abstract. Machine Translation (MT) is receiving increasing attention within 
language education due to both its affordances and the potential it offers for 
academic misconduct. To understand more about teachers’ views and practices, 
a survey was conducted with 153 foreign language (L2) educators who teach at 
Japanese universities. The survey examined their; (1) use of MT to assist with L2 
reading, writing, speaking, and listening, both personally and in their courses; (2) 
views on MT in language education; and (3) knowledge of how to help students use 
MT effectively and their willingness to learn about this. The results indicated that 
most teachers supported the use of MT as a learning tool, but many had concerns 
over its misuse. Few felt they had enough knowledge to guide students toward 
effective use and the vast majority wanted to learn more about how to do so. These 
findings highlight a need for teacher support and underscore the necessity of more 
research and workplace discussions on the integration and regulation of MT within 
L2 education. 


Keywords: machine translation, online translator, language learning, L2 teacher 
practices. 
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perceived potential for misuse (Kennedy, 2022). Research has pointed toward 
deficiencies in teachers’ awareness of student use (Alm & Watanabe, 2021), 
so it is important to learn more about MT from teachers’ perspectives. This 
study aims to understand how language teachers view and use MT. Personal 
experiences can influence teacher cognition, so teachers’ own use of MT and 
their integration of it into their courses were explored. The central Research 
Questions (RQs) are as below. 


RQ1: How do language teachers use MT, both personally and in their 
courses? 


RQ2: What are teachers’ views on the use of MT in the context of L2 
education? 


RQ3: Do teachers feel they have enough knowledge to help students use 
MT effectively to develop their L2 skills or do they want to learn more 
about this? 


2. Method 


Data were collected through an online survey (supplementary materials) created 
with Testmoz. This survey targeted foreign language teachers at Japanese 
universities. Calls for anonymous volunteers were posted in Facebook groups 
for Japan-based teachers (Online Teaching Japan; JALT SIGs). The RQs were 
addressed through closed and Likert-scale Survey Questions (SQs). The survey 
also gathered background information about participants and invited them to take 
part in follow-up interviews for a later phase of the study. SQs were developed 
by the researcher and refined with feedback from two reviewers. One reviewer 
was selected for their expertise in MT research to increase content validity. Raw 
statistical data were converted to percentages for descriptive analysis. 


Most of the 153 participants were L1 speakers of English (134) and Japanese (11) 
or both (1). There were also L1 speakers of the following languages: Indonesian, 
Portuguese, Italian, Urdu, Punjabi, Hindi, Filipino, Catalan, Spanish, French, 
German, Turkish, and Hungarian (including those with multiple L1s). Most taught 
English as a foreign language (147), three taught English plus another language 
(French, Spanish, Italian), and three taught other languages (Indonesian, Spanish, 
German). The majority spoke or studied L2s, with only four monolinguals. 
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3. Results and discussion 


There were 153 valid responses to the survey. All figures and tables draw on the 
full sample, with percentages rounded to the closest whole number. 


RQ1 explored how teachers use MT personally and as educators. Results showed 
teachers were familiar with MT through personal usage (Figure 1, SQs 5-8). 
Regular usage (very often/often) in their personal lives was far more common for 
reading 64% (98/153,) and writing 43% (66/153,) than listening 5% (7/153) and 
speaking 6% (9/153). 


Results revealed that teachers helped students use MT to develop reading and 
writing skills more than listening and speaking skills (Figure 2, SQs 9-12). This 
reflects their own usage patterns of MT to some extent, so familiarity as users may 
have influenced their teaching practices. However, a comparison of Figure | and 
Figure 2 shows that teachers’ integration of MT support into language courses was 
much lower than their personal use. 


Teachers were also asked about their experiences of discussing (un)acceptable use 
of MT with their students and setting usage guidelines. 21% (32/153) did it in all 
of their language courses, 51% (78/153) did it in some, and 28% (43/153) did not 
do it in any (Figure 3, SQ17). 


In summary, teachers’ personal use of MT was widespread, but far fewer introduced 
it to students as a learning tool. Many addressed appropriate use and offered usage 


guidelines, but few did this in all courses. 


Figure |. Teachers’ personal use of MT in their L2s (N=153) 
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Figure 2. Teachers’ MT support for students (N=153) 
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Figure 3. Level of teachers’ guidance on acceptable MT usage (N=153) 
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RQ2 investigated teachers’ views of MT in L2 education. First, the survey 
gauged teachers’ views on MT’s effectiveness as a tool for L2 skill development. 
Table | (SQs 18-21) shows that most felt it was effective (strongly agree/agree) 
for writing 71% (108/153) and reading 67% (102/153). Only a minority viewed 
it as effective (strongly agree/agree) for speaking 37% (56/153) and listening 
36% (55/153). 


Table 1. Perceptions of MT’s effectiveness for L2 skill development 


Strongly Agree Neither Disagree Strongly 

agree disagree 
Writing 27% (42) 43% (66) 22% (33) 7% (11) 1% (1) 
Reading 22% (34) 44% (68) 24% (36) 9% (14) 1% (1) 
Speaking 4% (6) 33% (50) 43% (66) 15% (23) 5% (8) 
Listening 5% (7) 31% (48) 44% (68) 14% (21) 6% (9) 
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Table 2 (SQs 13-14) shows teachers’ perceptions of their students’ use of MT. 
Most thought it was used in beneficial ways, with 71% (109/153) believing at 
least some students (all/most/some) used it to learn their L2. However, there 
was substantial concern over misuse, with 55% (84/153) believing at least some 
students (all/most/some) used MT to cheat. Interestingly, while 11% (17/153) 
felt all students used it to learn, only 1% (2/153) felt it was universally used for 
cheating. This suggests that teachers are certainly concerned over misuse but 
tend to view MT as more of a support tool than a cheating aid. 


Table 2. Teachers’ perceptions of students’ use of MT 


All Most Some A few None No 
students |students | students | students thoughts 
Perceptions of 11% 24% 37% 24% 3% 3% 
student use of MT (17) (36) (56) (36) (4) (4) 
for L2 learning 
Perceptions of 1% 14% 40% 26% 14% 5% 
student use of (2) (21) (61) (40) (21) (8) 
MT to cheat 


Despite concerns over inappropriate use, few teachers strongly agreed (1%) or 
agreed (16%) that MT use for coursework should be very restricted or banned 
(Table 3: SQ15); 20% were undecided and 62% (95/153) were in disagreement, 
suggesting that banning MT is not the way forward. However, most participants 
strongly agreed (48%) or agreed (39%) that it is essential for teachers to discuss 
(in)appropriate use of MT with students in all language courses and provide 
guidelines for acceptable use (Table 3: SQ16). This shows a willingness for 
educators to accept MT in L2 education provided there is guidance. Nevertheless, 
Figure 3 indicated only 21% of teachers actually did this in all courses, illuminating 
a gap between beliefs and practices that warrants further examination. 


Table 3. Teachers’ perceptions on the need to restrict MT usage and provide 


guidelines 
Strongly | Agree Neither Disagree | Strongly 
agree disagree 
1. MT should be 1% 16% 20% 44% 18% 
heavily restricted (2) (25) (31) (67) (28) 
or banned 
2. MT usage guidelines 48% 39% 12% 2% 0% 
are essential in all (73) (59) (18) (3) (0) 
language courses 
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RQ3 investigated teachers’ knowledge of how to help students use MT to develop 
L2 skills and their willingness to learn about this. Table 4 (SQs 22-23) shows that 
only 31% (48/153, strongly agree/agree) felt they had sufficient knowledge to help 
students use MT effectively. The vast majority (84%, 129/153, strongly agree/ 
agree) wanted to learn more, highlighting a need for teacher training materials and 
programs. 


Table 4. Teachers’ evaluation of their MT knowledge and willingness to learn 


more 
Strongly | Agree Neither Disagree | Strongly 
agree disagree 
1. Have enough 5% 27% 22% 41% 7% 
knowledge to help (7) (41) (33) (62) (10) 
students use MT 
2. Want to learn more | 37% 48% 12% 3% 1% 
about how to help (56) (73) (18) (5) (1) 
students use MT 


4. Conclusions 


This study indicated that teachers’ personal use of MT, their beliefs on its 
usefulness, and integration of MT support into their courses aligned somewhat, 
with reading and writing ranking higher than listening and speaking in all three 
areas. However, teachers’ inclusion of MT in their courses was more limited 
than their private use of these tools. There was concern over MT’s potential as 
a cheating aid, but few teachers wanted it heavily restricted or banned. Despite 
widespread agreement on the necessity of discussions and guidelines for 
appropriate use, only a minority had addressed this in all courses. Furthermore, 
many felt they had insufficient knowledge to help students use MT effectively 
for L2 skill development, and the vast majority wanted to learn more about this. 
These findings signal a need for increased discussion within L2 educational 
settings, greater teacher awareness and support (training materials/programs), 
and more research in the field of MT in L2 education. 


5. Supplementary materials 


https://research-publishing.box.com/s/hkpqr2t68zxu4qq0bqgjivb 1 wwer9qdh 
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Using the bilingual Corpus of Romanian 
Academic Genres (ROGER) platform 
to improve students’ academic writing 
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Abstract. Learner corpora of written texts from academic writing assignments 
provide a practical resource for students, particularly in fostering academic writing 
skills. One such corpus is the newly available ROGER (Corpus of Romanian 
Academic Genres), a bilingual comparable corpus containing learner discipline- 
specific academic writing data in Romanian native language (L1) and English as 
a foreign language (L2). This paper aims to illustrate a series of academic writing 
teaching approaches supported by the ROGER platform (launched in May 2022) 
to be applied by tutors in an academic writing classroom setting. The results are 
structured according to Adel’s (2010) methodological model for fostering rhetorical 
functions and specific phraseology in academic writing, coupled with addressing 
metadiscourse markers to better assist in the enhancement of students’ academic 
writing skills at the university undergraduate level. 


Keywords: academic writing, bilingual corpus, corpus-based teaching, learner 
corpora. 


1. Introduction 


While access to general and specialised corpora has improved significantly in 
recent years, corpus-based teaching in academic writing has yet to be exploited 
consistently in different student groups. One such case pertaining to teaching 
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English for academic purposes is employing learner corpora, which directly 
contribute to improving second language teaching and learning through classroom 
use (Granger, 2002). Learner academic corpora may equally constitute a precise 
reference point and a practical resource for students (Timmis, 2015), particularly 
in the context of fostering academic writing skills. However, learner corpora 
of academic writing are limited in availability: they are frequently not publicly 
accessible due to copyright laws, commercial reasons (Nesi, 2012), or emerging 
data protection regulations around the world (such as the General Data Protection 
Regulation in the EU). 


Bilingual learner corpora of academic writing are even less common, even though 
their added pedagogical benefit is significant. Specifically, these corpora provide 
students with the opportunity to discover interlanguage interference in a corpus of 
texts that are at the same level as their own. Previous studies (Bercuci & Chitez, 2019; 
Chitez & Bercuci, 2019) have shown that, in a Romanian higher education context, 
such corpora can be utilised as support for self-directed learning and are a source of 
motivation for students. As far as Romanian L1 implications are concerned, using 
corpora for academic writing is a complete novelty. Instruments such as academic 
phrase banks or academic word lists, already developed for English, are non-existent 
(see Rogobete, Muresan, & Chitez, in press) because Romanian students’ writing 
skills in L1 are taken for granted. At the same time, a marked tendency towards 
lingering under the influence of stereotypical language structures has been observed 
and demonstrated (Tucan, Pop, Bercuci, & Chitez, 2020). 


Figure 1. ROGER platform interface (https://roger-corpus.org/) 
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Given the limited availability of bilingual learner corpora of academic writing, a 
research project was launched at the West University of Timisoara, Romania, in 
2017, aiming to produce the first bilingual Romanian and English learner corpus of 
academic writing texts from both undergraduate and graduate students in a variety 
of disciplines. ROGER (Chitez et al., 2021), with a dedicated open-source query 
platform (Striletchi, Chitez, & Csiirés, 2022), was made available to the public in 
late May 2022 (Figure | above). This paper aims to illustrate a series of academic 
writing teaching approaches supported by the ROGER platform. 


2. A methodological model 


A useful methodological model of employing learner corpora to foster academic 
writing skills can be found in Adel (2010), who directs attention toward rhetorical 
functions and specific phraseology as essential features to be mastered by 
novice academic writers. In what is dubbed as a ‘hands-on’ model, Adel (2010) 
recommends addressing the following research questions to students using learner 
corpora in an academic writing classroom setting: 


“What do academic writers say when they...? (a) give an example, (b) 
refer to other texts or researchers, (c) introduce the topic, (d) start their 
Conclusion section?” (p. 43). 


The investigation of these questions through a learner corpus reveals hitherto 
unnoticed specific phrases and patterns in rhetorical functions and academic 
phraseology. The replication of this ‘hands-on’ model may be undertaken by 
employing the ROGER platform, with specific results showcasing salient 
dimensions of Romanian L1 and English L2 learner academic writing. Also, 
aside from Adel’s model, students may discover metadiscourse markers that are 
erroneously used in L2 by extracting word and N-gram lists in both languages. 
In teaching activities, students may detect the overuse of specific metadiscourse 
markers by using an English L1 corpus (such as BNC, COCA, or MICUSP) as a 
reference point. 


3. Results 


(a) When inputting the Romanian ‘de exemplu’ (253 occurrences), one notices that 
the phrase may be encountered either in sentence-initial or mid-sentence positions, 


317 


Alexandru Oravitan, Madalina Chitez, Loredana Bercuci, and Roxana Rogobete 


with a wide variety of punctuation patterns: while it is usually preceded by a 
comma or a full stop, it is followed by an array of punctuation marks, which can be 
used to address punctuation issues in student writing. 


(b) In the case of referring to other texts, materials, or researchers, the parallel 
querying of the corpus with the study level filter at the undergraduate level 


showcases significant differences which can be exploited pedagogically (Table 1). 


Table 1. Examples of in-text referencing from the ROGER corpus 


RO BA BA BA EN BA SY 1) BA BA 
SY1 SY2 |SY3 SY 2 SY3 

potrivit lui 3 0 2 according to 93 61 42 

conform lui | 8 0 4 

afirm* ca 34 4 40 state* that 27 20 28 

considera ca | 34 3 31 considers that (4 7 4 

prezentat* in | 8 1 18 present* in 23 38 35 

mentionat* in | 2 1 mention* in 8 15 25 

preluat* din 2 1 quoted in 0 1 

citat* in/ din | 0 0 cited in 0 3 3 


(c) The introduction to any academic paper highlights several phrases which can 
be elicited by students, particularly in connection to the thesis statement. Thus, 
students may be asked to predict patterns headed by ‘this paper *’ for English and 
‘aceasta lucrare’ for Romanian, and to compare and contrast their examples with 
concordance lines obtained when querying the corpus (Figure 2 below). 


(d) As concerns the insights that students might gain from how academic writers start 
their conclusion section, the querying of the keywords ‘concluzie’ for Romanian 
(271 occurrences) and ‘conclusion’ for English (511 occurrences) illustrates that 
both of these structures are usually preceded by a preposition or preposition plus 
article combination (‘in’/’ca 0’ for Romanian, ‘in’/’as a’ for English), followed by 
a comma and a summarising/concluding statement. 


In the case of metadiscourse markers, a persistent structure used in Romanian 
academic writing is ‘din punct de vedere’. In the ROGER corpus, the phrase 
appears 20 times both in Romanian and in English. However, in English L2, the 
phrase ‘point of view’ is often overused in the following configuration: ‘from a * 
point of view’ by Romanian native speakers. 
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Figure 2. Search results for ‘this paper’ (EN) and ‘aceasta lucrare’ (RO) in the 


ROGER corpus 


1 


Search results 


Texts displayed on page: 1- 20/123 


Language. English | Genre: Essay | Discipline: Humanities (HUM) | Study levef Master | Study year 
this paper==>... including both specialists and common people. In this paper, | will try to look at the impact that immigration... [shaw more] 


Language. English | Genre: Essay | Discipline: Humanities (HUM) | Study levet Master | Study year.1 
this paper==>... are race neutral (). Given the context, the aim of this paper is to provide a description of how this... [show mare] 


Language. English | Genre: Essay | Discipline: Humanities (HUM) | Study /evet Master | Study year.1 
this paper==>... discussed in the modern times. Therefore, the aim of this paper is to show that, despite its theoretical... [show more) 


Language. English | Genre: Essay | Discipline: Humanities (HUM) | Study levet Master | Study year. 2 
This paper==>... through her perspective as a young Southern girl This paper focuses on discussing To Kill a Mockingbird... [show more] 


Language. English | Genre: Essay | Discipline: Humanities (HUM) | Study levet Master | Study year 
this paper==>... opportunity and reverse discrimination The purpose of this paper is to study how the use of race conscious... [show more] 


Language. English | Genre: Thesis (Bachelor) | Discipline: Humanities (HUM) | Study levet Master | Study year1 
this paper=: framework that! will be making use of throughout this paper. As already stated above, my purpose is... [shaw more] 


Language. English | Genre: Thesis (Master) | Discipline. Computer Science (COMP) | Study level Master | Study year. Unknown 
this paper==>... ones retrieved online are appropriately cited in this paper. respecting the rules of avoiding Copyright... [show more] 


Search results 


Texts displayed on page: 1- 20/54 


6 


ine, Political Scie 


uncil), Aceasta lWcrare artistica publicitaré a lor, 


non-profit NROC( Natural Resources Defense Co nore] 
Language: Romanian | Genre: Thesis (Bachelor) | Discipline Political Sciences (POL) | Study fevet Bachelor | Study year, 3 
aceasta lucrare==>... studiu. Finele incheagé informatiile redactate in Beasts luerare printr-o concluzie cu referire... [show more} 
Lanquage: Ramanian | Genre: Thesis (Bachelor) | DF 1e Political Sciences (POL) | Study level Bachelor | 


Aceasta lucrare==>.. PUBLICITATEA LA PRODUSELE DE LUX ABSOLVENT: , INTRODUCERE Acgasta luerare are ca tema Forme ale... [show mare 


w 


Romanian | Genre Thesis (Back Discipline. Humanities (HUM) | Si 


aceasta lucrare==>.. ESTETICA RENASTERII ABSOL 


ve. Political Sciences (POL) | Study fevel Bachelor | Study year. 3 


esis (Bachelor) | Discipline Political Sciences (POL) | Stuc 


Language: Romanian | Ge 


aceasta lucrare==>_. operationare in institutie. Cre nore] 


yage:Ramanian | Genre: 


aceasta lucrare==>.,. de trecut, Obiectivul studiului de caz abordat in identia... [show more] 


Language: Romanian | Genre. Referat | Discipline Political Scier dy year. 3 


aceasta lucrare==>... publicului specialist, cat si a publicului general, in aeeesta lUctare imi propun sa descriu imaginile.. [show more] 


4. 


Discussion and conclusion 


The approaches presented in this short paper illustrate how the ROGER platform 
becomes an important resource for showcasing key academic phraseology in 
authentic bilingual contexts. Thus, valuable insights may be gained regarding 
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punctuation errors in (a) and (d), cases of cross-contamination (such as ‘considera 
ca’ — ‘considers that’ and ‘prezentat* in’ — ‘present* in’), which may be studied 
separately, but also situations that could be approached in individual concordance 
lines to observe citation and acknowledgement practices (b), or frequent action 
verbs and patterns in use when writing thesis statements (c). 


The results are structured according to Adel’s (2010) methodological model for 
fostering rhetorical functions and specific phraseology in academic writing and 
suggest adding another function addressing metadiscourse markers to better assist 
with enhancing undergraduate university students’ academic writing skills. In 
sum, we recommend the use of bilingual learner corpora as a tool for improving 
and enhancing academic writing skills, thus building a legitimate contrastive 
pedagogical model. 
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Pronunciation practice in Google Translate: 
focus on French liaison 


Kevin Papin! and Walcir Cardoso” 


Abstract. This paper examines the impact of self-directed pronunciation practice 
using Google Translate (GT) on the acquisition of French liaison (the production of 
latent consonants when they appear in consonant-vowel contexts across words: /z/in 
le/za/mis ‘the friends’). Second-language (L2) French learners completed homework 
activities on GT to practice pronunciation, utilizing the tool’s Text-To-Speech (TTS) 
and Automatic Speech Recognition (ASR) capabilities. Using a pretest/posttest 
design, the acquisition of liaison was assessed in terms of phonological awareness, 
perception, and production. The results indicated that while the L2 French learners 
significantly improved in their ability to produce liaison, their awareness and 
perception were not affected due to ceiling effects. Overall, learners reported 
positive views of their GT-based, self-directed learning experience. 


Keywords: Google Translate, automatic speech recognition, text-to-speech 
synthesis, L2 pronunciation. 


a IE Introduction 


Speech technologies have been under increasing scrutiny from Computer Assisted 
Language Learning (CALL) researchers, particularly for their potential to aid 
pronunciation instruction (Derwing, Munro, & Thomson, 2022). In CALL, TTS 
synthesizers and automatic ASR have been incorporated into L2 pronunciation 
teaching to provide machine-generated models to learners, who can then practice 
their pronunciation by having the software detect their speech and convert it to 
text (Cardoso, 2018). While empirical research suggests that the use of TTS and 
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ASR can support L2 pronunciation development in English (e.g. Cardoso, 2018; 
McCrocklin, 2016), only a handful of studies have targeted L2 French (Liakin, 
Cardoso, & Liakina, 2017). Moreover, two studies have examined the benefits of 
combining TTS and ASR in a single learning experience (He & Cardoso, 2021; 
van Lieshout & Cardoso, 2022). GT is one application in which these speech 
capabilities can be explored for pronunciation instruction. 


To examine the use of GT and its built-in speech capabilities for teacher-guided 
but autonomous pronunciation instruction, we focused on the acquisition of a 
phonological feature of French: the liaison, or the re-syllabification of latent 
consonants when they appear in consonant plus vowel contexts (e.g. compare 
/z/ in liaised ‘le/z/ amis’ [le.za.mi] ‘the friends’ with non-liaised ‘le/z/ livres’ [le. 
li.vr] ‘the books’). This mixed-method study addressed two Research Questions 


(RQs). 


RQI: Does the pedagogical use of GT’s speech features (TTS, ASR) as 
part of a homework assignment assist in the learning of French liaison? 


RQ2: What are the L2 learners’ views on the proposed GT-based 
pronunciation practice? 


2. Method 


Twenty adult L2 French learners (beginners) participated in this case study. 
Adopting a perspective to L2 research that reflects ‘pedagogical realities’ 
(Erlam & Tolosam, 2022), participants were asked to complete a self-directed 
pronunciation practice using TTS and ASR in GT as part of a one-hour homework 
activity. The activity targeted French liaison, a phonological phenomenon 
that poses difficulty to beginner learners, and consisted of three learner- 
paced components. First, learners were asked to copy-and-paste 13 sentences 
containing liaisons in GT by pressing the speaker button (TTS) to listen to the 
synthesized voice (1 in Figure 1). Second, learners were instructed to click on 
GT’s microphone to orally produce the same sentences for ASR conversion to 
text (2 in Figure | below). 


To measure the impact of the homework practice on the participants’ learning 
of liaison (RQ1), a pretest was administered before the GT practice, which was 
followed by immediate and delayed (two weeks later) posttests. These online 
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tests included multiple-choice questions based on audio recordings, and the 
learners’ audio recordings of target phrases containing the liaison. Participants’ 
phonological development was assessed across three levels of analysis, following 
Celce-Murcia, Brinton, and Goodwin’s (2010) framework for pronunciation 
instruction: phonological awareness (e.g. Are [target sentences] pronounced the 
same? Describe), perception (e.g. Listen and check if you hear an example of 
‘liaison’ or ‘no liaison’), and production (e.g. Read aloud sentences [containing 
liaison]). A focus group discussion was also held among four participants to gain 
insights into their views (RQ2) of the proposed pedagogical experience (e.g. Do 
you feel your French pronunciation improved using GT?). 


Figure 1. Homework activity sample: GT interface 


%p Texte B Documents ® Sites Web 
DETECTER LA LANGUE FRANCAIS ANGLAIS ARABE v - ANGLAIS FRANCAIS ARABE v 
Aprés une petite pause, une question va étre x After a short break, a question will be asked to x 
posée aux étudiants. the students. 
gu (@) 1 65/5000 . «) OD % < 
%p Texte B Documents @ Sites Web 
DETECTER LA LANGUE FRANCAIS ANGLAIS. ARABE v e ANGLAIS FRANCAIS ARABE v 
Parlez maintenant Traduction 
©? vm 


3. Results 


3.1. Learning gains 


No significant changes were observed for awareness, F(2, 38)=.95, p=.621, partial 
n2=.03, or perception, F(2, 38)= 2.26, p=.20, partial n2=.11. However, the proposed 
intervention led to statistically significant changes in production over time, F(2, 
38)= 11.40, p<.001, partial n2=.38. Post hoc analysis with a Bonferroni adjustment 
was conducted, showing a significant increase between the mean scores on the 
pretest and delayed posttest, 3.04, (95% CI, 1.72 to 4.36), p<.001, d= 1.35. Table | 
reports the evolution of test scores between the pretest and the two posttests, 
while Figure 2 provides a visual representation of these results, emphasizing the 
significant values obtained for production. 


324 


Pronunciation practice in Google Translate: focus on French liaison 


Table 1. Test scores: French liaison 


Pretest Immediate posttest | Delayed posttest 

M SD M SD M SD 
Awareness (/3) 2.55 0.61 2.55 0.51 2.70 0.47 
Perception (/16) 9.20 2.35 9.75 1.94 10.40 1.96 
Production (/18) 4.64 3.42 5.78 3.73 7.68 3.88 


Figure 2. Production scores (converted to %): French liaison 
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3.2. Learners’ views 


Content analysis of the focus group interview indicated that interviewees found 
their experience ‘very useful’ overall. Two learners explained that the homework 
activity ‘helped [them] pay more attention’ to the liaison and ‘raise [their] 
awareness’ to this phonological feature. GT was perceived as a ‘robust tool’ for 
L2 pronunciation learning, as the TTS feature can be used to easily check ‘how to 
pronounce’ words. Participants felt that the ASR feature has the potential to help 
them to self-correct, as ‘pronunciation must be perfect’ to be properly detected 
by the ASR. Regarding the drawbacks of the experience, two learners found it 
‘annoying’ and ‘discouraging’ when the ASR could not detect their speech. This 
sparked comments such as ‘I honestly don’t plan to [dictate to GT]’ and it is ‘so 
more useful for GT to be reading things to me [than me speaking to it]’. After this 
self-directed pronunciation practice, learners reported they felt encouraged to 
continue to employ the newly acquired pronunciation learning strategies in their 
own learning (e.g. to check pronunciation using TTS). However, they did not 
fully value the pedagogical use of the tool’s ASR capability, despite the learning 
gains described earlier. 
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4. Discussion and conclusions 


This paper provided initial empirical evidence that the pedagogical use of GT’s 
speech features (TTS and ASR) can support the development of liaison in L2 
French (RQ1). While the proposed intervention led to a statistically significant 
improvement on the production of liaison, it had no impact on awareness and 
perception. This could be attributed to a ceiling effect in both awareness and 
perception, as the learners already had advanced knowledge of liaison before the 
experiment (2.55/3 and 9.2/16 on the pretest respectively, leaving little room for 
improvement), compared to production (4.64/18 on the pretest). These results are 
in line with the predictions embedded in Celce-Murcia et al.’s (2010) framework 
for teaching pronunciation, which posits that L2 phonological features should 
be taught in the order in which they are acquired, starting with the development 
of phonological awareness, then progressing toward perception and production. 
Due to the initial level of liaison knowledge of our participants in awareness and 
perception, it is not surprising that the proposed treatment only affected their 
production. Teachers willing to integrate GT to their pronunciation teaching 
should therefore be mindful of their students’ initial phonological knowledge 
and use GT to target the stage(s) that need(s) improvement. 


Due to the naturalistic and self-regulated setting adopted in this study, some 
external variables might have negatively affected the participants’ performance in 
completing the activities (e.g. internet connection speed, ambient noise at home). 
The limited number of participants and the fact that this study was based on a 
one-time intervention should also be acknowledged as a limitation. However, this 
is also one of the strengths of our study, which constitutes a ‘pedagogical reality’ 
(Erlam & Tolosam, 2022) with ecological validity, involving an activity that is 
common among L2 learners: the completion of out-of-class, teacher-guided, self- 
regulated homework assignments. Future studies should examine the impact of 
GT-based pronunciation practice on the development of different phonological 
features of French (and other languages) and explore learner-GT interactions to 
examine behavior-effect relations. 
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Abstract. The present exploratory study investigated the efficacy of secondary face- 
to-face classroom-based English as a Foreign Language (EFL) instruction digitally 
supported by the FeedBook, an interactive computer assisted language learning web- 
based suite of exercises providing item-level scaffolded feedback. Seventy-seven 
native (L1) German seventh-grade students used the FeedBook during four two-week 
training periods (cycles, here we analyse Cycle 2 and 3). Classroom and FeedBook 
practice occurred in parallel, except for grammar constructions for which only 
FeedBook practice was provided (controls). At the end of Cycle 3, students engaged 
in a classroom-based communicative task for which the practised constructions 
were relevant. Custom-designed pre- and post-tests, administered via the FeedBook, 
assessed language accuracy in each cycle. Mixed-effect models revealed significant 
pre-/post-test accuracy gains independent of the learners’ proficiency in English, 
but not in controls. Gains from digitally supported instruction were also positively 
related to accurate use of EFL grammar constructions in the communicative task. 
Overall, the results indicate learning benefits for face-to-face classroom-based 
instruction supported by the FeedBook, evidenced both in grammar tests and in 


communicative activities. 


Keywords: digital EFL learning, linguistic skill transfer, ICALL feedback, 
communicative language teaching. 
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Enhancing EFL classroom instruction via the FeedBook... 


4: Introduction 


A large body of classroom-based research, both paper-based and digital, has 
highlighted the key role of practice in foreign language (L2) learning (DeKeyser, 
2015; Pili-Moss, Brill-Schuetz, Faretta-Stutenberg, & Morgan-Short, 2020). 
In the present study, classroom-based instruction and practice inspired by task- 
based language teaching principles were combined with digital practice via the 
FeedBook, a web-based workbook providing scaffolded interactive feedback 
on linguistic forms and meaning. We will refer to such face-to-face classroom 
instruction complemented by individual practice using digital tools as ‘hybrid’ 
instruction. 


The FeedBook is fully integrated with the curriculum and covers different 
grammar foci, vocabulary, different language skill areas (listening comprehension, 
reading comprehension), and exercise types (Meurers et al., 2019, pp. 164-165). 
Consistent with research findings highlighting the potential efficacy of digitally- 
mediated EFL instruction (Chong & Reinders, 2020; Schmidt & Strasser, 2022), 
previous FeedBook studies employing a pre-/post-test design have also found 
that the use of the platform contributed to a significant increase in language 
accuracy based on results from the post-tests, particularly when feedback was 
provided (Meurers et al., 2019). 


Building on these findings, we not only looked at the efficacy of hybrid instruction 
as measured in post-test accuracy gains, but also explored relationships between 
test outcomes and proficient use of L2 English grammar constructions in a 
communicative task (i.e. a task designed to create communication conditions 
similar to those in immersive contexts) performed at the end of the instruction 
cycle. The research questions were formulated as follows. 


RQ1. To what extent is face-to-face classroom instruction complemented 
by individual classroom-based practice using digital tools (i.e. hybrid 
instruction) related to significant language accuracy gains? 


RQ2. Are there any benefits for digital practice alone? 
RQ3. To what extent are language accuracy gains in hybrid instruction 


related to accurate use of grammar constructions in face-to-face classroom- 
based communicative activities? 
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2. Methods 


2.1. Participants and study design 


Seventy-seven seventh graders (level A2 of the CEFR; native German learners 
of L2 English; 41 females, 36 males; three intact classes) from one inner-city 
secondary school in Northern Germany participated in the study. Data were 
collected over the course of a school year from September 2021 to June 2022, 
during which students received regular classroom-based EFL instruction in 
English (three 45-minute classes a week). Students additionally engaged in 
individual classroom-based grammar practice through the FeedBook (Meurers et 
al., 2019), a web-based workbook providing a suite of grammar exercises with 
scaffolded interactive feedback on linguistic forms and meaning, over four two- 
week practice cycles (about 2.5 hours per week) focusing on a set of grammar 
constructions (Figure 1). 


Figure 1. Overall study design with data subset analysed in the present study 
(yellow) 


Classroom teaching 


Classroom-based end-of-cycle tasks (audio- 
visual data) 


Future 
Relative Pr. 


Pre Post 


FeedBook Log data 


For the purposes of the present study, only pre-/post-test data from Cycle 2 
(conditionals and comparatives) and Cycle 3 (direct questions) were used, as well 
as data from the written task at the end of Cycle 3 (supplementary materials S1). 
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2.2. | Measures and data analysis 


At the beginning and at the end of each cycle, students completed custom-designed 
pre- and post-tests administered via the FeedBook (30 fill-in-the-gap sentences 
each) assessing accuracy in the relevant grammar constructions. In Cycle 2, one 
construction (conditionals) was taught and practised face-to-face in the classroom, 
as well as practised on the FeedBook, whereas the other (comparatives) was only 
practised on the FeedBook. Language proficiency in the written task at the end of 
Cycle 3 was measured by creating a composite index obtained by standardising 
and averaging four components: (1) number of questions produced, (2) number of 
correct questions using the past tense, (3) number of correct questions using the 
do/does/did auxiliaries, and (4) number of incorrect questions (a negative index). 
The main inferential analysis employed mixed-effects models controlling for 
random effects of participants, test items, and the class group the students belonged 
to on intercepts, as well as multiple regression. A two-level categorical variable 
identifying test type (pre-test/post-test) was the main predictor, whereas students’ 
L2 English proficiency was used as a covariate in all analyses. Descriptive statistics 
relative to test scores in Cycle 2 and 3 are reported in Table S2.1a (supplementary 
materials $2), whereas Table S2.1b reports students’ mean scores in English and 
proficiency in the Cycle 3 target task. 


3. Results 


To answer RQI and RQ2, data from Cycle 2 were analysed to explore pre-/post- 
test accuracy gains in three different cases: overall for Cycle 2 (Table 82.2, 
supplementary materials $2), only for items testing the conditional tense, and only 
for test items testing the comparative (Table |). To answer RQ3, a regression model 
was used with accuracy gains in Cycle 3 as a main predictor and proficiency in 
English questions in the classroom-based task as the outcome variable (Table $2.3; 
supplementary materials $2). 


Table 1. Pre-/post-test accuracy models: C2 conditional and C2 comparative 


Wald CI (95%) 
Fixed effects |B SE Z lower upper Pp 

o, | (Untercept) 0.89 0.41 2.16 0.83 1.71 .031* 

fo] 

g 

‘§ | Pre-test -1.37 0.30 -4.48 -1.95 -0.76 <.001*** 
= 

g 

5 Eng Prof -0.45 0.12 -3.79 -0.68 -0.22 <.001*** 


331 


Diana Pili-Moss et al. 


8 (Intercept) 2.08 0.88 2.39 0.34 3.81 .019* 

g Pre-test -1.40 0.85 -1.65 -3.06 0.26 099 

2, 

5 Eng Prof -0.84 0.22 -3.87 -1.26 -0.41 <.001*** 


Note. *p <.05; ***p <.001. °N of observations=2,826. +N of observations=1,352. 


4. Discussion and conclusion 


The first two research questions asked to what extent classroom-based, face-to-face 
instruction that is supported by the FeedBook is related to a positive development 
in EFL language outcomes (RQ1), and whether accuracy gains also emerge for 
grammar constructions for which only FeedBook practice was provided (RQ2). 
The analysis showed that for both overall C2 scores and for items testing the 
conditional alone, post-test scores were significantly higher than pre-test scores. 
Furthermore, these results were obtained after controlling for differences in L2 
English proficiency. This indicates robust positive effects of the combination of 
classroom-based, face-to-face instruction and FeedBook practice for all students, 
independent of ability. 


The second linguistic target covered in Cycle 2 (comparatives) was practised only 
digitally on the FeedBook. In this case, the analysis (Table 1) showed that pre- 
/post-test gains did not emerge as statistically significant, when controlled for 
English proficiency. This suggests that English proficiency was the main factor 
driving accuracy in this case, rather than an effect of stand-alone digital instruction. 
Note that this was the case although the time dedicated to FeedBook practice was 
substantial and exceeded the time spent in face-to-face instruction. 


The third research question explored the relationship between language accuracy 
gains in questions (Cycle 3) and accurate use of questions in a written classroom- 
based communicative task. The results of a regression model (Table S2.3; 
supplementary materials $2) show that stronger improvements in the accuracy 
of questions in tests were significantly related to a greater ability to apply the 
acquired knowledge in communicative tasks, and that the relationship obtained 
was independent of the students’ proficiency in English. 


Overall, our findings highlight clear positive advantages for the combination 
of traditional, classroom-based face-to-face instruction with classroom-based 
individual practice using digital tools, which did not emerge if the latter was 
not aligned with curriculum topics covered face-to-face in the classroom. 
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Moreover, the benefits emerged for all pupils independent of their proficiency in 
L2 English, suggesting that integrating classroom-based face-to-face instruction 
with classroom-based individual digital practice has the potential to become an 
inclusive educational strategy which may prove particularly effective with mixed- 
ability groups. 


5. Supplementary materials 
S1: https://research-publishing.box.com/s/Si0y6xrllk3r5e6y3p4mfmxpS5irpxpcw 


82: https://research-publishing.box.com/s/getzwbt74y7w8k3noyxpn80i5z3pi6hf 
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Computer-assisted pronunciation training 
in Icelandic (CAPTinI): developing a method 
for quantifying mispronunciation in L2 speech 


Catlin Richter!, Branislav Bédi’, 
Ragnar Palsson’, and Jon Gudnason* 


Abstract. We are developing a Computer-Assisted Pronunciation Training (CAPT) 
system for learners of Icelandic as a second and foreign language (L2). Based on 
pre-designed tasks in pronunciation exercises, this system will provide corrective 
feedback on learners’ speech. One of the main features we are implementing is a 
new method for automatic pronunciation scoring to provide immediate feedback 
on learners’ errors. We report promising results for a pilot study of this method 
in Norwegian, where the pronunciation score successfully distinguishes between 
native speakers and adult learners, and we discuss how this method informs our 
continued development of Icelandic CAPT. 


Keywords: pronunciation training, computer-assisted pronunciation training, speech 
error detection, pronunciation scoring, Icelandic as a second or foreign language. 


4. Introduction 


Currently, there are no mobile applications, websites, or other online tools on 
the market that enable learners of L2 Icelandic to practise real speaking skills. 
According to a recent review of online tools for L2 Icelandic, there are about 
15 tools offering pronunciation exercises or theoretical explanations about 
pronunciation to learners (Bédi, 2022). The available online tools that aim to help 
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learners with practising spoken Icelandic offer only pre-recorded sounds of letters, 
words, phrases, or shorter texts with full sentences which learners can listen to 
and, if they wish, repeat aloud. As a result, no corrective feedback about learners’ 
mispronunciation is provided. 


However, current technology for delivering such feedback has unsatisfactory 
performance, even in the most commonly taught L2s such as English. Using state- 
of-the-art methods to detect L2 pronunciation errors, only six of ten identified 
‘errors’ were actually mispronounced (i.e. 60% precision), while just 40% to 80% 
of incorrect pronunciations are detected (i.e. recall; Korzekwa, Lorenzo-Trueba, 
Drugman, & Kostek, 2022). Furthermore, these technologies depend on excellent 
Automatic Speech Recognition (ASR) or Text-To-Speech (TTS) support for the 
target language, so performance grows even worse in languages without highly 
developed ASR or TTS. 


Therefore, high-quality automatic pronunciation feedback for Icelandic must be 
created. This article presents development towards a system for CAPT in Icelandic 
(CAPTinI) that will deliver such feedback to learners, including a method to 
immediately detect mispronounced parts of learners’ productions, and an initial 
evaluation of this method in a related language. 


2. CAPT design 


The CAPTinI system is applied as part of a series of lessons which give the 
learners practice with selected examples of vowels, consonants, words, phrases, 
and sentences. As such, the system simulates a classroom setting although only 
virtually, and corresponds with Crabbe’s (2003) learning-opportunity framework, 
which includes elements of comprehensible input and output, interaction 
exercises, feedback, and rehearsal opportunities, all of which lead to language 
understanding and possibly learning. The lesson content and sequencing 
coordinate with the course design of Icelandic Online, a freely available web- 
based course series developed by the University of Iceland and launched in 2004, 
which has about 80,000 active users (Arnbjérnsdottir, Fridriksdottir, & Bédi, 
2020). In our interactive pronunciation exercises, learners at different levels 
may listen to correct pronunciations and repeat them, or attempt to read text 
aloud without errors, and in either case continue practising with feedback after 
each attempt until they achieve accurate pronunciation. We first used Norwegian 
learners’ speech to validate the method for detecting pronunciation accuracy, and 
it will be integrated with CAPTinI later. 
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3. Scoring method 


We propose the RelativeDTW method of scoring pronunciation accuracy, using 
Dynamic Time Warping (DTW) to quantify how closely learners’ speech matches 
both native (Ll) and other L2 speakers. DTW measures similarity between 
samples of speech by aligning and comparing corresponding elements. It 
accurately quantifies L2 accent strength by comparing L2 speakers with a set of 
L1 (‘reference’) speakers. DTW has previously been applied in different ways for 
other CAPT systems (Bartelds, Richter, Liberman, & Wieling, 2020; Yue et al., 
2017) and therefore is suitable here. 


To help factor out confounds, RelativeDTW also compares the learner’s 
pronunciation to a set of L2 references and determines pronunciation accuracy 
by a difference-to-sum ratio of the two DTW scores (Figure 1). This reflects 
the observation that accurate pronunciations are relatively closer to L1 than L2 
pronunciations, regardless of absolute DTW values, so the ratio facilitates consistent 
mispronunciation detection across learners. Similar two-way comparisons are 
often beneficial for CAPT (Fu, Chiba, Nose, & Ito, 2020; Jia et al., 2014). 


Figure 1. Equation for RelativeDTW score: the difference-to-sum ratio of DTW 
values from L2 and L1 speakers’ reference recordings 


DTWL2-DTWL1 
DTWL2 + DTWL1 


RelativeDTW = 


4. Experiment and results 


Since DTW is comparative, evaluating a learner’s utterance requires reference 
recordings of other speakers saying the same sounds or words. While data collection 
for these Icelandic recordings is still in progress, we performed a pilot study 
validating the method in Norwegian using the NB Tale corpus*, which includes 
240 Norwegian L1 speakers representing maximum dialect diversity and 117 L2 
speakers from various backgrounds. 


RelativeDTW is evaluated on ability to distinguish between native and non- 
native speakers’ pronunciations, so the overlapping coefficient (OVL) of score 


5. NB Tale — Speech Database for Norwegian https://www.nb.no/sprakbanken/en/resource-catalogue/oai-nb-no-sbr-3 1/ 
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distributions for L1 versus L2 test recordings expresses the method’s effectiveness. 
To estimate practical utility, we also report recall for detecting L2 speech with the 
score threshold set to correctly accept at least 95% of L1 speech. As a baseline 
we evaluate the method of Bartelds et al. (2020), measuring DTW distance to L1 
references only, using wav2vec-2.0 speech representations for all evaluations. 


Results in Table | show that Ll and L2 speakers receive quite different 
pronunciation scores from our method; some individual L2 phonetic segments are 
indistinguishable from L1 speakers’, but errors contribute to overall non-native- 
like accents in longer speech. In all cases, RelativeDTW outperforms the baseline. 


Table 1. Evaluation of how effectively our RelativeDTW score and the Bartelds 
et al. (2020) baseline distinguish L1 and L2 speakers in the NB Tale 
corpus 

Overlap L2 recall at 95% L1 acceptance 
RelativeDTW Baseline RelativeDTW Baseline 
Sentences | 0.10 0.33 97% 68% 
Words 0.34 0.62 63% 30% 
Phonemes | 0.45 0.76 46% 17% 


5. Discussion and conclusions 


The RelativeDTW difference ratio is sensitive to non-native-like pronunciations 
of L2 speakers, without rejecting valid L1 variation, and it performs better than 
the Bartelds et al. (2020) baseline. Although our task of classifying L2 but not L1 
speakers as ‘non-native-like’ is not directly comparable to other previous work 
(e.g. Korzekwa et al., 2022), we can report 90% precision with 46% recall for 
phonemes, or 93% precision with 63% recall for words, i.e. 63% of non-native 
Norwegian speakers’ words were identified as sounding non-native-like while 
under one in ten of so-called ‘mispronunciations’ were native speakers. These 
results use 25-speaker reference sets, but even reduction to three speakers identified 
mispronunciations in 28% of L2 words (OVL=0.54). 


The NB Tale speech database provides a challenging first test for RelativeDTW, 
because these Norwegian learners are already advanced and fluent while the native 
speakers’ dialects are highly diverse. Icelandic contains far less dialect variation, 
and CAPTinl is integrated with lessons for less advanced learners, so these pilot 
results provide assurance that RelativeDTW should perform well in Icelandic 
once we complete data collection. The requirement for reference recordings, and 
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consequent impossibility of scoring spontaneous speech, is a main limitation of 
this method. However, no other corpora in the target language are needed, so it 
is feasible even for languages with limited existing speech technology. Other 
pronunciation scoring methods are more flexible at the cost of needing hundreds of 
hours of training data or specialist annotation (Korzekwa et al., 2022). 


Currently, we have begun implementing scoring for a selection of Icelandic 
pronunciation exercises, and will continue development and evaluation with this 
new dataset. We are collecting more Icelandic recordings through the Samromur 
platform, the main speech data gathering platform for Icelandic language 
technology. Across all projects, 4,000 hours of speech from nearly 30,000 L1 and 
L2 Icelandic speakers have been collected so far. Our preliminary results using 
Norwegian samples promise a positive language-learning experience in CAPTinI 
for improving learners’ pronunciation in L2 Icelandic. 
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Exploring the impact of a group dynamics 
training activity on learner engagement during 
online classes of French as a foreign language 


Ana Rosso!, Jackie Robbins’, and Christine Appel? 


Abstract. This paper focuses on the impact of introducing a Group Dynamics 
Training Activity (GDTA) on learners’ reported engagement during the course. The 
context is a university Common European Framework of Reference for languages 
(CEFR) B2 French language online course over the course of two semesters. Data was 
collected in the form of online surveys during the semester prior to the introduction 
of the GDTA and during the semester when it was administered. This case study 
uses quantitative analysis of Likert-scale question responses and qualitative analysis 
of open fields in the questionnaires using a content analysis methodology. Results 
indicate that the GDTA had a positive impact on learners, particularly on the social 


and cognitive dimension of learner engagement. 


Keywords: collaboration, engagement, affective, cognitive, social, online. 


le Introduction 


The Centre for Modern Languages (CIM) at the Open University of Catalonia 
offers fully online language courses. Learners work in virtual classrooms where 
they interact with their teacher and peers. The methodology is task-based learning 
and includes collaborative work, a compulsory element of the course’s continuous 
evaluation. 
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The literature on collaborative work provides a theoretical justification to its 
inclusion in course design: it promotes meaningful learning (Karppinen, 2005), 
can foster motivation (Rogat, Linnenbrink-Garcia, & DiDonato, 2013), and 
increases language learning gains (Hsu & Lo, 2018). These benefits can promote 
increasing learner engagement, a construct comprising three dimensions: social, 
affective, and cognitive (Svalberg, 2009). The social dimension refers to learners’ 
participation and involvement in pedagogical tasks, the affective aspect represents 
their emotional connection to their learning and peers, and the cognitive element 
revolves around what is learned and how. 


Ensuring learners become and stay engaged is a priority for language courses, and 
particularly important in entirely online courses which can have high dropout rates 
and in which learners have reported feeling isolated (Yuan & Kim, 2014). While 
collaborative work can help learner engagement, anecdotal experience at the CIM 
has shown it can have the opposite effect due to issues such as conflict within 
groups, scheduling and time management difficulties, and unequal participation 
amongst group members, also reported in studies with a similar scope (Sun, 2014). 


In order to enable learners to fully benefit from peer collaboration, we designed a 
GDTA to train them to overcome issues related to negative dynamics within learner 
groups working collaboratively. 


This case study seeks to understand the impact of a GDTA on learner engagement 
in terms of learners’ sense of connectedness to their peers, motivation, and 
perceived learning gains. Below, we describe the methodology and report on the 
results according to the three dimensions of learner engagement — social, affective, 
and cognitive. 


2. Method 


The case study’s context is a fully online upper-intermediate (B2 CEFR) French as 
a foreign language course offered biannually at the CIM. Learners take the course 
as part of their undergraduate degree and are aged 24 to 56. 


Data was collected from anonymous and voluntary questionnaires sent to two 
cohorts from two consecutive semesters (Table 1). In total, 69 learners responded 
to the survey. Gender information was not collected. Both cohorts participated in 
the same course activities, with the difference that the GDTA was implemented 
only for the second one. 
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Table 1. GDTA, number of classrooms, and learners 
GDTA #Classrooms #Learners #Survey 
enrolled responses 
1. Cohort 1 No 5 149 39 
2. Cohort 2 Yes 122 30 


The collaborative activities consisted of pair and small group (three to four 
learners) activities, using Google Docs for asynchronous collaborative writing in 
groups, and a videoconferencing tool for synchronous pair speaking. For Cohort 1, 
small groups and pairs for these tasks were formed ad hoc before the start of the 
activities, and might stay the same if the learners wanted, but they might also 
change for each task. 


For Cohort 2, the GDTA was implemented before beginning collaborative work. 
This activity consisted in taking part in a group Skype call to get to know each other 
better, and plan for upcoming collaborative tasks. The planning included assigning 
roles to different group members, discussing the steps involved in the writing task, 
as well as forming pairs for the speaking tasks and agreeing on meeting times. 
Instructions were provided to guide the Skype meeting conversation, which was 
recorded and sent to their teacher. 


Two online surveys were administered to both cohorts: the first survey before 
the start of collaborative tasks and the second survey at the end of the course. 
Both surveys included three seven-point Likert-scale items asking learners about 
their engagement with the course, each corresponding to a dimension of learner 
engagement: “I feel connected to my classmates” (social), “I am highly motivated in 
this course” (affective), and “I am improving my French” (cognitive). Additionally, 
the end of semester survey included two open questions: “Which aspect of pair and 
group work motivated you the most” (affective), and “which activity (collaborative 
writing or pair speaking) helped you learn the most” (cognitive)? Quantitative 
analysis of the Likert-scale questions and qualitative analysis of open fields in the 
questionnaire were conducted using a content analysis methodology. 


3. Results and discussion 


3.1. | Quantitative analysis of Likert-scale answers 


« “T feel connected to my classmates” 


342 


Exploring the impact of a group dynamics training activity... 


Cohort | and 2 learners’ responses indicated a rise in their connection with their 
peers after collaborative work (groupwork). For Cohort 2, this increase was slightly 
higher, something which could be attributed to their participation in the GDTA 
(Figure 1). 


Figure 1. Perceived connectedness to classmates before and after groupwork for 
Cohort | and Cohort 2 


Connectedness: Cohort 1 before and after groupwork (n=39) Connectedness: Cohort 2 before and after groupwork (n=30) 
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¢ “Tam highly motivated in this course” 


Compared to Cohort 1, answers from Cohort 2 indicated a decrease in learners’ 
reported level of motivation, which can perhaps be explained in more depth when 
looking at the answers to the open question on motivation (Figure 2). 


Figure 2. Perceived motivation before and after groupwork for Cohort 1 and 
Cohort 2 


Motivation: Cohort 1 before and after groupwork (n=39) Motivation: Cohort 2 before and after groupwork (n=30) 
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¢ “Tam improving my French” 


The highest increase for both cohorts was in learners’ perceived learning gains 
before and after collaborative work. However, after the GDTA this was slightly 
higher for Cohort 2 (Figure 3). 


343 


Ana Rosso, Jackie Robbins, and Christine Appel 


Figure 3. Perceived learning gains before and after groupwork for Cohort | and 


Cohort 2 
Perceived learning gains: Cohort 1 before and after groupwork (n=39) | Perceived learning gains: Cohort 2 before and after groupwork (n=30) 
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3.2. Qualitative analysis of open questions 


Here, learners shared many thoughts often unrelated to the actual question, which 
nonetheless helped yield insights into their ideas and experiences about the course. 
These answers were then coded according to positive and negative comments. 


Issues related to time management, scheduling, and unequal participation were 
reported by Cohort |. Negative comments from both cohorts mentioned perceived 
level differences amongst group members/pairs preventing participants from 
performing to the best of their abilities, and the importance of assessment, given 
the context. While nine Cohort | learners expressed negative emotions, ranging 
from ‘stress’ to ‘panic’, only one Cohort 2 student commented on the ‘coldness’ 
of peer interactions. In Cohort 1, two learners described working collaboratively 
as motivational, an idea which was reinforced by Cohort 2, where nine learners 
commented on this. 


In Cohort 2, four learners mentioned the continuous collaboration and interaction 
with group members as positive, reinforcing the idea that the GDTA and the ability 
to work within the same group of learners increased their feelings of connectedness 
to their peers. 


Very few positive emotions were reported by Cohort | learners: four learners 
highlighted that working in groups allowed them to get to know others. However, 
they did not connect positive emotions with motivational or learning gains, in 
contrast with learners in Cohort 2, where words like ‘motivation’ or ‘motivated’ 
(5), and ‘enriching’ (1) were connected with positive emotions, such as ‘happy’ (4), 
‘enjoy’ (2), and words related to peer connection, such as ‘breaking the ice’, and 
“making friends’ (3). Despite a decrease in reported motivation on the Likert-scale, 
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open answers from Cohort 2 indicated learners connected motivation to perceived 
learning gains arising from such connections. Furthermore, while four learners in 
Cohort | indicated a preference for working alone, only one learner from Cohort 
2 did. 


4. Conclusions 


The Likert-scale responses suggest bigger differences in the dimension of cognitive 
engagement. However, the qualitative analysis reveals positive emotions and an 
increased sense of connectedness that learners identified as instrumental to their 
learning after GDTA. These results, although tentative given the limited number 
of participants and scope of analysis, highlight the importance of including GDTA 
before the start of collaborative work to help learners achieve a more positive 
learning experience. Although the differences are not substantial enough to draw 
firm conclusions, the positive results warrant further exploration, which should 
include classroom participation logs analysis and a comparison of perceived 
learning gains, and learner assessment results. 
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online teaching and face-to-face on-campus 
teaching using the DDL approach in L2 
English verb-noun collocations 
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Abstract. This study explores the effectiveness of Data-Driven Learning (DDL) 
approach to second-language (L2) English vocabulary learning in on-demand 
online distance learning at a private university in Tokyo, Japan. The participants 
were 49 Japanese undergraduates, intermediate L2 English learners at the B1 level 
in the Common European Framework of Reference for languages (CEFR). The 
experimental group consisted of 25 students who completed verb-noun collocation 
correction tasks between the pre-test, and the post- and delayed post-tests. The 
control group consisted of 24 students who took the tests but did not perform the 
error correction task. The results show that for memorizing verb-noun collocations, 
on-demand online DDL — in which learners refer to screenshots of concordance lines 
—is as effective as DDL in a traditional on-campus setting, where learners search the 
Corpus of Contemporary American English (COCA) for themselves, even though 
the learning activities in remote online on-demand and face-to-face on-campus 
DDL are not exactly the same. This study suggests the potential of DDL in online 
on-demand classes. When a variety of teaching methods are being explored, DDL 
seems a viable option as an approach to L2 teaching. 


Keywords: data-driven learning, remote teaching, collocations, error correction. 
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1. Introduction 


DDLis an inductive discovery learning method in which learners search for words 
and grammatical items in a corpus and observe examples of their usage; it has been 
reported to be effective in L2 learning with vocabulary acquisition (e.g. Boulton, 
2017). While the teacher, author of this article, has used a DDL approach in face- 
to-face English classes for about a decade, she used it for the first time in on- 
demand distance learning classes in 2020 due to the COVID-19 pandemic. This 
study examines the effects of the DDL approach on L2 vocabulary learning in 
remote online on-demand classes by comparing them with those in Satake (2021), 
which was conducted in face-to-face classes on-campus. In this study, a remote 
online on-demand course refers to a course in which the author uploads self-study 
materials to the university’s Learning Management System (LMS), learners submit 
assignments, and the teacher provides feedback. 


2. Method 
The procedure for this study is as follows: 


¢ pre-test (five minutes, Week 1, both the experimental and the control 
groups); 


* two verb-noun collocation correction tasks (Weeks 3-9, the experimental 
group only); 


* post-test (five minutes, Week 10, both the experimental and the control 
groups); 


¢ delayed post-test (five minutes, Week 15, both the experimental and the 
control groups); and 


* analysis. 


Participants were 49 first-year students majoring in history at a private university 
in Tokyo, Japan, who took a remote online on-demand English course taught 
by the author in the fall semester in 2020. All students agreed to voluntary 
participation in this study. The participants were intermediate L2 English learners 
at the BI level in CEFR and had been studying English at school for eight years. 
They were 37 male students and 12 female students, aged 18-20, and all were 
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native Japanese. The experimental group consisted of 25 students who completed 
verb-noun collocation correction tasks. They took three tests: the pre-test was 
given two weeks before the first task, the post-test the week after the final task, 
and the delayed post-test one month after the post-test; 19 learners took all three 
tests, so only their data were used. The control group consisted of 24 students 
who took only the pre-, post-, and delayed post-tests, but did not perform the 
error correction task; the author used the data from the 21 learners who took 
all three tests. In each test, the 14 target verb-noun collocations were presented 
in one sentence each, and the verb parts were fill-in-the-blank questions. The 
following is an example: 


( ) a deep breath and relax. 


The same questions were used for all three tests. The learners were given five minutes 
to complete each test. If a verb in parenthesis was contextually appropriate, it was 
not required to be a target collocation verb, and it was considered to be correct. 
To verify the validity of the tasks, an analysis of variance test was conducted on 
the results of the pre-, post-, and delayed post-tests for both the experimental and 
control groups. 


For the tasks, the learners were provided with a handout and were asked to read 
two English sentences containing highlighted unnatural verb-noun combinations 
with unnatural verb choices, which the author selected (here, ‘unnatural’ means 
extremely low frequency). They corrected these sentences by referring to the corpus 
resources, then transcribed the referenced example sentences in English, wrote up 
their findings in English or Japanese, and submitted their learning record through 
the university’s LMS. No explicit instruction on appropriate collocation was given 
after the task. The learners completed 14 sets of verb-noun collocation learning 
tasks, two tasks each week for seven weeks. Table | below shows the 14 target 
collocations that the learners were likely to use in their writing assignments in 
the textbook for the course because they are listed as vocabulary to be learned in 
the textbook. In unnatural verb-noun combinations, ‘do’ was always used, since 
the unnatural combinations would be natural in Japanese if they were literally 
translated into Japanese. The following is an example of highlighted unnatural 
verb-noun combinations; the underlined words were originally highlighted: 


Do a deep breath and relax. 


For the qualitative analysis, the author consulted the worksheets submitted by the 
learners in the experimental group. 
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As reference materials, screenshots of the concordance lines were used by the 
learners (e.g. see Figure 1) because the classes were conducted as on-demand 
distance classes, which did not allow sufficient instruction for the learners to use the 
corpus directly and appropriately. The author searched the COCA corpus for verbs 
co-occurring with the nouns of the target verb-noun collocations and distributed 
the screenshots of concordance lines in which the target nouns co-occurred with 
three high-frequency verbs, including the correct verb. As a reference corpus, 
the author used the COCA, a large balanced corpus with an easy-to-use interface 
(Davies, 2008). 


Table 1. Target collocations 


1 take a breath 8 make a statement 

2 take control of one’s life 9 give a speech 

3 make a photocopy 10 make a change 

4 make a guess 11 take an action 

5 take the lead 12 make a contribution 
6 | give a welcome 13 give a performance 
7 make an argument 14 make progress 


Figure 1. Example of screenshots of the concordance lines 


[4] Corpus of Contemporary American English [gs @ @ Fo iB 


SEARCH FREQUENCY OVERVIEW 
ICLICK FOR MORE CONTEXT a) CHOOSE LIST |---| CREATE NEW LIST ful ‘SHOW DUPLICATES 
1 2012 BLOG blog. mozilla.org 4A BC. warrant being jerk in forum, # Pull your head out and take a deep breath of fresh air. Mozilla does not need to explain why they pulled the update 
2 2012 | BLOG aj-eon.com ABC. . Make one thing happen. And when you're done, take a deep breath and do it agein, About POE # | used to be an unremarkably 
3 2012 BLOG kidneysandeyes.com AB C_ well, she wanted to know what was up. # | took a deep breath, looked at Julian and we started to tell the story to Quinnlin with trepidation 
4 2012 BLOG thoughtcatalog.com |A BC. soonas read this, | put the book down and took a deep breath in. Isn't that something? The whole point! Amy Poehler, a 
5 2012 BLOG jihadwatch.org AB C_ thathe is the best suited person for the job. # Take a deep breath with me. Now exhale. Do you really feel less secure with Barack Obama 
6 2012 BLOG ..llance:wordpress.com A BC to go somewhere | don't want to go. # | took a deep breath and looked out of the window.” Stop being stupid. "| mumbled 
7 2012 BLOG ..llance.wordpress.com A B C_ lace one more time. # | closed my eyes, and took a deep breath. | could do this. | just needed to concentrate on Jess, on 
8 2012 BLOG ..lisonvesterfel.com | BC. .1do know what to do. just need to take a deep breath and gather the courage to do it. This is definitely one of your 
8 2012 BLOG wetpaint.com A BC. Klaine and Brittana fans in particular might want to sit down and take a deep breath. Brad says, " Long-distance relationships are hard. " He continues orr 


10 2012 BLOG ..enathoughtblooms.com A B C_ be able to lift the lid and get my keys. # Taking a deep breath, | plunged my hand... then arm into the depths of the trash can 

12 2012 BLOG ..nwatch.wordpress.com A B C_ before you publicly speculate about people's careers and judgment, take a deep breath and ask yourself who the hell you think you are to so freely besm 
13 2012 BLOG otherfood-devos.com A B C_ even ministry and Christian service. Amen. #" That you took a deep breath an hour ago won't be much appreciated unless you're still breathing. A 

14 2012 BLOG angela.cabeen.net A BC goes, then I'm really going to freak out. # Take a deep breath... you can do this... BLEH!! | MISSED!!! 

15 2012 BLOG cookiesandclogs.com AB C_ fun’ inwriting. Starting now, I'm going to take a deep breath and try to get my creative juices going again. # | have a few 


16 2012 BLOG thebittenword.com AB _C_Dressing using a recipe from the final issue of Gourmet (everyone take a deep breath and think happy thoughts). We were surprised to find that we loved 


3. Results 


As Figure 2 shows, the experimental group showed an increase in the post-test 
and delayed post-test scores over the pre-test with the mean pre-test score of 
4.3, the mean post-test score of 8.6, and the mean delayed post-test score of 6.8, 
while the control group showed only a minimal change between tests with the 
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mean pre-test score of 4.0, the mean post-test score of 4.2 and the mean delayed 
post-test score was 4.7. As the results of the analysis of variance show, there were 
significant differences in between-group main effects with large effect size (F(/, 
119)=11.879, p=.001, 7? =.17), between-test main effects with medium effect size 
(F(2, 119)=52.864, p<.001, 7? =.12), and interactions with medium effect size 
(F(2, 119)=40.365, p<.001, 7° =.09). The results of the sub-effect test revealed 
significant differences between the groups for the post-test (experimental group 
> control group, p<.01) and the delayed post-test (experimental group > control 
group, p<.01). Significant differences were also obtained between the tests for the 
experimental group (post-test > pre-test, p<.01; post-test > delayed post-test, p<.01 
delayed post-test > pre-test, p<.01). Since the post-test and delayed post-test scores 
were significantly higher than the pre-test scores only in the experimental group, it 
can be said that DDL, in which learners refer to screenshots of concordance lines 
in remote classes, is effective for learning verb-noun collocations. 


Figure 2. Pre-, post-, and delayed post-test results for experimental and control 
groups 


10- 
ae | Experimental 
* 2 contro! 


1 2 3 
Pre-test Post-test Delayed post-test 


4. Discussion and conclusions 


This study demonstrates that learning verb-noun collocations in remote on-demand 
classes by referring to screenshots of concordance lines may be as effective as 
learning to search for verb-noun collocations by learners in face-to-face classes 
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on-campus (Satake, 2021). Since the learners did not search the corpus themselves 
but only read the concordance lines selected by the teacher, it is similar to paper- 
based DDL, which uses printed concordance lines selected by the teacher, and thus 
there is more teacher intervention and less learner autonomy. The results confirm 
previous research that computer-based and paper-based DDL are equally effective 
as L2 learning tools (e.g. Chujo & Oghigian, 2012). Since the DDL in this study 
does not have an autonomous search, learning may have a qualitative difference 
from face-to-face DDL in on-campus classes. The learners in this study often 
made simple statements such as “I found that this verb comes before this noun”, 
while the face-to-face DDL learners in on-campus classes often described deeper 
discoveries about the meaning of the verb in the collocation when comparing 
and determining which verb was appropriate. Qualitative differences in learning 
could affect the method and amount of output of learned vocabulary, the use of 
associative knowledge, and other aspects of learned knowledge. However, the 
results show that simply looking at the given search results without doing a search 
is effective for the purpose of long-term memory of vocabulary items. This study 
suggests the potential of DDL in distance learning, which has been accelerated as a 
new teaching method during the COVID-19 pandemic. Where a variety of teaching 
methods are being explored, DDL is a viable option as an approach to L2 teaching. 


5. Acknowledgments 


This work was supported by JSPS KAKENHI Grant Number 20K 13109. 


References 


Boulton, A. (2017). Corpora in language teaching and learning. Language Teaching, 50(4), 483- 
506. https://doi.org/10.1017/S02614448 17000167 

Chujo, K., & Oghigian, K. (2012). DDL for EFL beginners: a report on student gains and views 
on paper-based concordancing and the role of L1. In J. Thomas & A. Boulton (Eds), /nput, 
process and product: developments in teaching and language corpora (pp. 170-183). 
Masaryk University Press. 

Davies, M. (2008). The Corpus of Contemporary American English (COCA). https://www. 
english-corpora.org/coca/ 

Satake, Y. (2021). The effects of DDL on learning verb-noun collocations in English. Proceedings 
of the JAECS Conference 2021. Japan Association for English Corpus Studies (pp. 85-90). 


351 


EU 


SpEakWise VR: exploring the use of social virtual 
reality in telecollaborative foreign language learning 
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Abstract. This paper discusses the potential of immersing foreign language learners 
in Social Virtual Reality (SVR) in an international seminar, designed by the authors, 
called SpEakWise VR, combining the tandem language learning paradigm with 
gamification in immersive virtual environments. SpEakWise VR builds on an 
existing telecollaboration (SpEakWise) between undergraduate students in Trinity 
College Dublin and Hildesheim University. We set out to explore how inclusion 
of an SVR activity influences student engagement, learning involving intercultural 
and multilingual problem solving, and team building. This paper presents initial 
observations from our research, concerning multimodality in embodied immersion 


and code-switching, and proposes future avenues of inquiry. 


Keywords: social virtual reality, digital game-based language learning, 


telecollaboration, pragmatic competence, embodiment. 


1. Introduction 


SpEakWise is a telecollaborative seminar running annually since 2007 
between Trinity College Dublin and the University of Hildesheim, involving 
undergraduate students of languages and intercultural communication at both 
universities. The seminar’s aim is to foster foreign language skills (German 
and English) and the development of pragmatic and critical cultural awareness, 
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and, with this, to promote intercultural competence (Martin, O’Sullivan, & 
O’Rourke, 2013). SpEakWise has been optimized in response to evolving 
technologies: in early iterations, students interacted via web chat and in recent 
years via videoconferencing. The most recent version has seen the integration 
of a collaborative activity in an SVR context, which is the focus of this paper. 
Our concept unites the idea of tandem language learning (Telles, 2015) with 
gamification (Reinders & Wattana, 2015) and insights from the embodied and 
‘empractic’, i.e. ‘hands-on’, character of VR experiences. 


SVR is a new social medium with growing relevance beyond universities and 
classrooms (Senkbeil, 2021). It offers an innovative way of bringing students 
together in virtual learning spaces and providing an experience of quasi- 
embodied co-presence across geographical distance, through VR avatars. With 
the help of the developer’s kit of the SVR platform RecRoom, we created virtual 
environments in which our students solve multilingual cooperative puzzles 
together to progress through an ‘escape room’-style adventure which challenges 
them to test and improve their communicative problem solving competencies 
in their target language. It functions at the intersection of telecollaboration and 
Digital Game-Based Language Learning (DGBLL), augmented by the experience 
of VR as a space to meet, communicate, co-orientate, and coordinate action 
(Senkbeil, 2021). 


As opposed to screen-based virtual environments, VR creates the feeling for its 
users that they are involved in an actual three-dimensional world existing beyond 
the physical space that they normally inhabit, defined as the sense of presence 
(Lombard & Ditton, 1997). Our pilot studies on communication in SVR show 
that such discourse functions differently compared to screen-based digital-social 
interactions, because VR arguably influences its user in a more intensive, more 
visceral way than other media (Senkbeil, 2021; Senkbeil, Ahlers, Lazovic, & 
Schweiger, 2020). VR users often feel that they embody the avatar whose first- 
person perspective they share, rather than merely controlling a character from a 
third-person viewpoint. This sense of embodiment can result in intensive forms of 
the ‘proteus effect’ (Yee & Bailenson, 2007): given a high sense of presence in a 
VR experience, users do not mimetically ‘act as if’ they are that avatar, they are 
that avatar in that particular time and space. 


As there exists only limited research on the integration of immersive environments 
in tertiary-level teaching, we are broadly exploring how the inclusion of an SVR 
activity impacts on student engagement and learning in situations involving 
intercultural and multilingual problem solving and team building. Our research 
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also seeks to contribute to theory building concerning the impact of embodied 
immersion on communication through SVR. 


For SpEakWise VR, we seek to exploit these effects, informed by insights from 
cognitive linguistics, particularly the theory of embodied cognition, highlighting 
that there are strong associations between language acquisition, space, and motor 
concepts (Barsalou, 2008; Pulvermiiller, Hauk, Nikulin, & IImoniemi, 2005). Our 
research furthermore involves theoretical-methodological models from linguistic 
pragmatics that emphasize ‘empractic’ language use (Biihler, 2011 [1934]), ie. 
linguistic action intertwined with non-linguistic activities. 


2. Design of tasks and implementation 


The SpEakWise VR environments are designed to meet the requirements of tandem 
language learning and gamification, and to utilize the embodied and empractic 
character of VR experiences as mentioned above. The environments have a modular 
and adaptable structure: two to four players can play two to four ‘levels’ together, 
each taking about five minutes to solve. The gamification elements involve ‘escape 
room’-style collaborative puzzles in relatively small areas, so that users are always 
able to see and hear each other. Each team is divided in two halves, separated by a 
fence in the environment. The fence cannot be traversed, but communication and 
vision across it are unimpeded. The information necessary to solve the puzzle is split 
between the areas on either side of the fence, meaning that students must engage in 
negotiation of meaning across languages and cultures to succeed in the task. As is 
usual in language tandems, SpEakWise VR students were given the guideline that 
they should try to speak 50% of the time in German and 50% in English. 


The 27 students participating in the SVR activity had already been engaging with 
each other over several weeks via videoconferencing. The activity involves two 
stages. In the first stage, groups of four students (two Dublin and two Hildesheim) 
engage in a brief ‘warm-up’ activity, learning to move and interact with the head- 
mounted displays and handheld controllers. The DGBLL activity immediately 
follows, in which the students encounter the problem solving situations. Each session 
lasts 20-30 minutes, is recorded, and subsequently transcribed following conversation 
analytic principles. Students complete debriefing interviews about their experience. 


As our research is at an early stage, we summarize our initial observations on the 
intersection of language practice and immersion in a virtual environment in the 
following section. 
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3. Initial observations from our research 


One striking effect of the observed SVR interactions has been the employment of 
all multimodal affordances provided by the technology to achieve communicative 
success (here: finding solutions to collaborative puzzles). Students use their 
virtual embodiment (i.e. the intuitive mapping of their physical motions onto 
their avatars) to co-orientate with their listeners, to align viewpoints, and to 
gesture directions or the shape of objects. In contrast to videoconferencing, 
however, students engage with each other not just linguistically but also 
(quasi-)physically in a 3D virtual space. This invites researchers and teachers 
to pay attention particularly to lexical fields connected to space and motion 
(prepositions, deictics), pragmatic patterns to create co-ordinated attention, and 
giving directions (Senkbeil et al., 2020). 


Further, while we did not police the guideline around language use strictly (see 
Section 2), we observed quite frequently intensive forms of code-switching. 
For example, an Irish student, following German instructions to deduce the 
number code which opens the door to the next level, explained to her German 
listeners: 


“the first number is three and then the second number is eins weniger als 
die erste Zahl, so it’s two” (Enl, SpEak Wise in-game corpus). 


Such cases demonstrate students’ involvement and intrinsic motivation to solve 
the task and continue their ‘adventure’. Learners creatively use all linguistic and 
multimodal affordances in — almost always — communicatively successful ways. 
The students themselves commented in debriefing interviews on code-switching 
as an effective means to an end. For example, a Trinity College Dublin student 
observed: 


“It was interesting with the languages, I thought, how we changed 
between... as far as I felt it didn’t feel as if one language was dominating. 
We were switching between German and English quite frequently. And 
it was a really nice way [to communicate]” (En3, SpEakWise debriefing 
corpus). 


More generally, the debriefing interviews indicate that students are motivated, 
engaged, and experience SVR as a positive addition to their language learning 
journey. 
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4. Next steps in the development 
of SpEakWise VR and conclusion 


Based on initial observations from our study, SVR seems to elicit the employment 
of all multimodal communicative affordances, particularly based on learners’ 
experience of a sense of virtual embodiment through their avatars. SVR provides 
the potential for an empractic, 1.e. ‘hands-on’ approach to certain lexical fields, 
e.g. motion verbs, locations, prepositions, and deixis. Our next steps include 
further upscaling of the immersive experience (more users simultaneously, longer 
immersion, a growing variety of communicative challenges, etc.). Our research will 
continue to employ conversation-analytical methods, e.g. turn-by-turn analysis of 
transcripts. Moreover, we will add empirical measurements of learners’ progress, 
and work with user feedback to further improve the seminar. 
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Comparing lexical complexity 
using two different VE modes: a pilot study 


Tomotaka Shiroyama! 


Abstract. The aim of this study is to compare how two different Virtual Exchanges 
(VE) contribute to language learning in a Japanese university context. Task-Based 
Language Teaching (TBLT) using VE has been a focus in this study. There are huge 
advantages of using technology in language teaching such as increasing language 
use opportunities, feedback, and others (Lee, 2016). However, many recent TBLT 
and VE studies have focused on students’ interaction in the classroom (Hagley, 
2020). This study investigated whether two different types of VEs have the potential 
to promote ‘real-world communication’ in a university context. The findings show 
that there was no significant difference regarding lexical complexity between 
both modes. The result suggests that both modes are unlikely to enhance students’ 
vocabularies. 


Keywords: virtual exchanges, task-based language teaching. 


1. Introduction 


VE has incorporated TBLT in some instances but others have focused more on 
supplementing classwork with VE to allow students to participate in international 
interactions. Hagley (2020) notes that unless VE is incorporated into the syllabus, 
“EFL often became an academic activity with few chances to use English in real- 
world communicative events” (pp. 74-75). Ribeiro (2016) argues that in studies 
on synchronous computer-mediated communication, there has not been enough 
attention paid to the interaction between Non-Native Speakers (NNS) and Native 
English Speakers (NES). To solve this, the current study focuses on interactions 
between NES and NNS using Synchronous VE (SVE), with an online chat, and 
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Asynchronous VE (AVE), with a forum discussion in the TBLT framework. This 
study refers to the question: 


RQ1. Are there any differences in lexical complexity between the discourse 
produced using SVE and AVE when NES interact with NNS? 


This research explores how VE can be utilized effectively by English instructors in 
the TBLT framework to enhance college students’ communicative abilities. 


2. Method 


2.1. Participants 


Eight participants took part in this research. They were all university students and 
over 18 years old and signed consent forms to take part in this study. The participants 
were divided into a target group of L1 Japanese students of English (N=7, one male 
and six females) and an L1 British English interlocutor participant. According to 
self-reported information in a background questionnaire, the target group were pre- 
intermediate (equivalent to TOEIC Listening and Reading 550), their mean age was 
20 years old (range 19-21), their average length of living abroad in English-speaking 
countries was one week (range zero to three weeks), and the average time spent 
studying English was 9.7 years (range six to 17 years). I targeted pre-intermediate 
students because most students in Japanese universities are pre-intermediate level. 


2.2. Instruments 


In this research, two decision-making tasks (see supplementary materials, 
Appendices A and B) and a background questionnaire (see supplementary 
materials, Appendix C) were used as research instruments. 


2.3. Data collection 


The data collections were conducted remotely several times on different dates. 
I divided the seven participants in the target group randomly into two groups, Group 
A and Group B. One interlocutor worked on tasks with seven students (three from 
Group A and four from Group B). Each student in the target group was asked to 
do two different decision-making tasks with the same interlocutor. The students in 
Group A were to work on a task for 20 minutes using SVE, the online chat (Zoom) 
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first, and then AVE, the forum discussion (Microsoft Teams). The Group B students 
were to work on a task using AVE, forum posting (Microsoft Teams) first, and then 
on another task using SVE, the online chat (Zoom). For the forum posting task, 
each group posted their opinions at least three times a week. In order to resolve the 
effect of posting order, the modes of VE and tasks are counterbalanced. 


3. Results 


3.1. Lexical complexity 


Lexical diversity is one of the measurements of lexical complexity. In the current 
study, I will follow McCarthy and Jarvis’s (2007) definition of lexical diversity as 
“the range and the variety of vocabulary deployed in a text by either a speaker or 
writer” (p. 459). 


To analyze lexical diversity, there are some measurements, for example the Type- 
Token Ratio (TTR), Guiraud’s Index (Root TTR), D (VOCD-D), and the Measure 
of Textual Lexical Diversity (MTLD). However, TTR, Root TTR, and VOCD-D 
are predisposed to be affected by text length (DeBoer, 2014). As a result, the 
current research employed MTLD, because this metric can measure various text 
lengths. The mean number of words written using SVE was 116.57 words per 
group, ranging from 59 to 177 words. The mean number of words produced using 
AVE was 123.28 words, ranging from 19 to 202 words. Therefore, texts varied 
considerably in length. To calculate the MTLD value, the computational software 
Coh-Metrix 3.0 was used. 


3.2. Results of lexical complexity between SVE and AVE 


In this research, eight participants took part in total. They were divided into two 
groups: a target group (N=7) and a group of intercalators (N=1) for the target 
group. The discourse of the target group was measured on the same dependent 
variable (i.e. the MTLD value) under two different conditions (i.e. SVE and 
AVE). For this research design, a paired-sample ¢-test was thought to be the most 
appropriate test. In order to judge whether the data set in the current research met 
with the assumptions of a paired-samples f-test, the difference between the two VE 
forms (SVE and AVE) was initially calculated. Then, the assumption of normality 
and no outliers was confirmed. As evaluated by inspection of a boxplot, outliers 
were not seen. The different scores were also normally distributed, as shown by 
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Shapiro-Wiki’s test, p=.063>.05. As a result, to examine differences using MTLD 
between SVE and AVE, it was calculated that a paired-samples ¢-test was the most 
appropriate statistical test. Descriptive statistics for MTLD in discourse in SVE 
and AVE are illustrated in Table |. The results of a paired-samples ¢-test indicated 
that there was no statistically significant difference between the MTLD scores in 
the two VE modes, #(6)=1.181, p=.282. 


Table 1. MTLD using SVE and AVE 


SVE (Chat) AVE (Forum) 

Mean iS) Mean |SD 
64.52 18.46 54.09 (12.63 
| Note. N=7. 


4. Discussion 


The current research was motivated by a desire to explore how different modes of 
VEs can be utilized effectively in the TBLT framework. Shiroyama (2021) argues that 
more than one mode of VEs is insufficient. Considering this, this research investigated 
the discourse features using two different forms of VEs and focusing on lexical 
complexity. Regarding task types, Shiroyama (2021) criticized that several studies 
used open-ended tasks, whereas this research employed decision-making tasks. 


Regarding lexical complexity, this study was not consistent with Hwang’s (2008) 
study. She states that the AVE group produced ‘a richer vocabulary’ as well as 
‘more complex written language’ compared with the SVE group. However, any 
comparison between this research and Hwang’s study requires caution. First, 
Hwang used different groups for the two modes. Consequently, the different result 
might have come from the difference in the participants in each group. Second, 
Hwang examined lexical complexity by analyzing lexical diversity as ‘the square 
root of the double number of tokens’ while this study employed lexical diversity 
calculated using the MTLD. The different types of activity (i.e. discussion in 
the case of Hwang’s research) may have affected lexical aspects of participants’ 
language production. 


5. Conclusions 


This research has examined the potential for the integration of different forms of 
VEs and TBLT. The finding in this research indicates that there was no statistically 
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significant difference regarding lexical complexity using SVE and AVE. For future 
research, firstly it will be necessary to examine not only lexical complexity and 
grammatical complexity points, but also fluency and accuracy to be able to effectively 
utilize VE within the TBLT framework. Another direction would be to focus on 
students’ language production when they work with different proficient interlocutors 
because today many young people tend to use language exchange applications i.e. 
Hello-Talk, Tandem. Conclusively, it is necessary to discover the ways in which VE 
can assist language learners to enhance their target language abilities. 
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The role of learners’ memory in app-based language 
instruction: the case of Duolingo 


Tro Vasileiou! and Diana Pili-Moss? 


Abstract. The current study investigated the role of visual short-term memory, 
working memory, and declarative memory as individual differences in the earliest 
stages of vocabulary and syntactic learning in Duolingo-based language instruction. 
Thirty-eight L1-Greek adults completed memory tasks and engaged in learning 
Navajo on Duolingo. Subsequently, vocabulary and syntax were assessed respectively 
via a word recognition, a word translation, and a grammaticality judgement task. 
Multiple regression analyses revealed an advantage for distributed practice both in 
vocabulary and syntax, after controlling for amount of practice. Further, declarative 
memory played a significant role in learning syntax and vocabulary, when measured 
in a word translation task. Extending the analysis for the first time to app-based 
environments, the results of the present study confirm the importance of declarative 
memory and distributed practice in adult acquisition of L2 vocabulary and syntax. 


Keywords: mobile-assisted language learning, Duolingo, individual differences, 


memory functions. 


“7 Introduction 


Ubiquitous availability of mobile-assisted language learning technology has led to 
the development of commercial tools, with studies relating their use to significant 
linguistic gains in second language (L2) vocabulary and grammar (Loewen et 
al., 2019). Mobile apps offer benefits including access to less commonly taught 
languages (Reinders & Benson, 2017) as well as features designed to increase 
learners’ motivation (Rachel & Rockinson-Szapkiw, 2018), such as experience 
points (Duolingo). Additionally, some apps (including Duolingo) provide 
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approachable bite-sized lessons offering adaptability to individual study habits 
(Rosell-Aguilar, 2018) and thus allowing a more distributed learning, a type of 
schedule for which advantages have been independently reported in the SLA 
literature (e.g. Bird, 2010). 


Next to learning schedule, many studies in the SLA literature have evidenced the 
role of cognitive individual differences including short-term and working memory 
(e.g. Dussias & Pinar, 2010). Additionally, experimental studies have reported that 
declarative long-term memory generally predicts L2 vocabulary learning and L2 
grammar learning in the early stages of acquisition (e.g. Morgan-Short et al., 2014). 
Although previous research has investigated the role of memory in computer- 
assisted environments, little research is available on the role of cognitive individual 
differences in more ecologically valid environments, such as commercial language 
learning applications. 


Considering the extensive use and success of Duolingo, our aim was to investigate 
the role of cognitive abilities in learning a new language through this application. 
The study’s main research question was formulated as follows: 


RQ: To what extent do working memory, visual short-term memory, and 
declarative memory predict vocabulary and syntactic learning in the early 
stages of Duolingo-based instruction? 


2. Methods 


2.1. Participants 


Thirty-eight L1-Greek adults (18 to 54 years of age) with no significant previous 
exposure to verb-final languages, including Navajo, agreed to learn the language 
on Duolingo for a period of five days. Greek was not available as a medium of 
instruction on Duolingo. However, participants’ proficiency (BI to Cl CEFR; 
based on self-reports) was considered sufficient for English to be used. 


2.2. Cognitive tests 


In Phase 1, participants completed three computer-administered cognitive tasks. 
Declarative long-term memory was assessed through a verbal paired-associates 
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task, visual short-term memory was measured via a visual forward digit span task, 
and working memory was measured via a backward digit span task. 


2.3. | Language learning on Duolingo 


In Phase 2, participants engaged in learning Navajo on Duolingo. The tracking 
feature Duolingo for Schools allowed researchers to assign participants the same 
set of lessons and review their practice schedules unobtrusively. 


Participants were assigned a total of nine lessons starting from the lowest level of 
difficulty from four thematic units (introduction, family, food, and animals), known 
as ‘bubbles’. The lessons included 46 lexical items (conversational expressions, 
nouns) and 16 sentences with four different verbs. Participants received no explicit 
rules but received correct/incorrect feedback and in some cases, the correct answer 
was presented after trial. The participants were free to distribute learning across the 
instruction period (one to five days). 


2.4. Linguistic assessment 


In Phase 3, one to two days after language instruction was completed, 
participants were given computer-administered tests to assess learning of Navajo 
vocabulary and syntax. For vocabulary learning, participants completed a word 
recognition and a word translation task. In the word recognition task, participants 
were asked to judge whether 16 presented words were Navajo words. In the 
word translation task, participants judged the correctness of 16 Navajo items 
paired with an English translation. For syntactic learning (word order learning), 
participants were assessed in a grammaticality judgement task, where they were 
asked to judge the word order grammaticality of 40 sentences. Grammatical 
sentences (20 items) followed the patterns typical of Navajo (SOV, Location-V) 
whereas ungrammatical sentences violated Navajo’s word order (*VSO, *SVO, 
*V-Location). 


3. Results and discussion 


Multiple regression models were calculated with accuracy in each assessment task 
as the dependent variable, and outcomes in the three memory functions as well as 
three co-variates as independent variables. The co-variates were: age at testing, 
number of days of Duolingo study (as a measure of distribution of practice), and 
L2 exposure (number of years of formal English instruction and residency in 
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English-speaking countries). A summary of the descriptive statistics can be found 
in supplementary materials S1. 


3.1. Vocabulary learning 


The first model analysed word recognition accuracy (Table 1) and revealed 
that no memory function predicted accuracy in word recognition. The only 
statistically significant (p=.001) predictor was the days of Duolingo study with a 
B value of .56. It is possible that the word recognition task required a relatively 
low cognitive load and thus differences in cognitive ability did not emerge as a 
significant factor. 


Table 1. Multiple regression model for word recognition accuracy 


Model |Predictors |B SE B Pp R? Adjusted R? 
1 361 .224 

STM .06 17 07 .626 

WM -.07 .19 -.08 .699 

DLTM 12 ll 18 295 

AaT .O1 04 04 821 

L2 Exposure — -.07 31 -.04 054 

Nr. Days ie 21 56 001 


However, memory functions held a predictive role in word translation accuracy 
(Table 2). 


Table 2. Multiple regression model for word translation accuracy 


Model | Predictors B SE B Pp R Adjusted R? 
2 433 .323 

STM 33 18 33 .038 

WM -.25 19 -.26 196 

DLTM 31 12 46 012 

AaT .00 04 .00 979 

L2 Exposure _ -.10 32 -.05 772 

Nr. Days 35 22 36 019 


Here declarative long-term memory was a statistically significant predictor 
(p=.010) with a correlation coefficient value of .46, and short-term memory 
was a significant predictor (p=.038) with a value of B=.33. These results are 
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in line with the hypothesis of a key role of declarative memory in learning of 
vocabulary, a relationship which might have also been strengthened by the 
pair-associative learning characteristics of the translation task. No significant 
relationship between working memory and word translation accuracy emerged. 
Once more, the days of Duolingo study was a significant predictor (p=.019) with 
a B value of .36. 


3.2. Syntactic learning 
The multiple regression model for syntactic accuracy (Table 3) revealed that 
declarative long-term memory was a significant predictor of accuracy (p=.007), 


with a B value of .53. 


Table 3. Multiple regression model for syntactic accuracy 


Model | Predictors B SE B Pp R Adjusted R? 
3 590 564 

STM -.36 45 -.12 439 

WM 45 49 .15 367 

DLTM 85 29 53 .007 

AaT -.21 .09 -31 .032 

L2 Exposure -1.7 83 -.30 .074 

Nr. Days .62 56 49 .008 


This finding is consistent with experimental studies that reported a role of 
declarative memory in grammatical accuracy in the early stages of adult 
L2 acquisition (Morgan-Short et al., 2014). Contrary to previous findings 
(Dussias & Pinar, 2010), working memory was not a significant predictor of 
participants’ sensitivity to Navajo sentence grammaticality (p>.05). Perhaps the 
role of working memory was not prominent as participants judged word order in 
sentences without complex structures which might have required less attentional 
resources. 


The days of Duolingo study was also a significant predictor (p=.008) with a 
coefficient of B=0.49. This finding shows that more distributed practice predicted 
higher syntactic accuracy scores, corroborating the findings of previous studies on 
syntactic acquisition (Bird, 2010). Age at the time of testing was also a significant 
predictor (p=.032) with a negative coefficient B=-0.3/, meaning that the older 
participants were, the more limited was their ability to distinguish grammatical and 
ungrammatical sentences in Navajo, possibly due to age-related cognitive decline 
(Salthouse, 2004). 
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4. Conclusions 


Overall, the findings of the present study suggest that declarative memory and 
visual short-term memory play an important role in lexical learning as measured 
in a word translation task. Declarative memory was also a significant predictor of 
syntactic learning. Moreover, the study reported a distributed study effect in both 
lexical and syntactic learning. This means that, when matched for the amount of 
instruction, the more days participants spent on the application, the better they 
performed in both vocabulary and grammar tasks. 


5. Supplementary materials 


https://research-publishing.box.com/s/z25f55ew3 wor8ekettiywe5n13j7uvij 
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Abstract. This paper explores and reports on teacher attitudes toward online 
assessment in foreign language teaching and learning in Spain and Finland. During 
the pandemic, teachers have had to face serious challenges, such as the method to be 
adopted, task design or handling student feedback, in order to ensure effective student 
assessment. They have had to find completely new ways to assess their students. 
A qualitative approach was adopted and data were gathered from a questionnaire 
shared on Google form, which was completed by 23 teachers from Spain and 11 
from Finland. The aim was to examine (1) the advantages and disadvantages of 
using online assessment tools to measure students’ progress, and (2) the participants’ 
perceptions of their experience of assessing their students online. The findings show 
that while online assessment tools, especially quiz apps, provide instant feedback 
and correction for students and teachers, it is difficult to control what tools students 
might be using to support their learning and/or assessment assignments. 


Keywords: assessment, online tools for language learning, teacher attitudes. 


1 Introduction 


Assessment plays a crucial role in all pedagogical programs, since it monitors 
students’ learning progress via activities which aim to measure whether the learning 
outcomes match the intended results of the educational program. In this regard, 
Webber (2012) points out that assessment implies “activities designed primarily 
to foster student learning” (p. 202). Consequently, it cannot be denied that, as part 
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of their professional development, teachers should design and use formative and 
summative assessment in their classroom. 


Online assessment of students has long been established because of the online 
learning mode. Nevertheless, several studies (e.g. Boitshwarelo, Reedy, & Bilany, 
2017; Brady, 2005) have reported problems that have emerged and affected their 
good practice. The shift from face-to-face conventional classroom assessment to 
an online environment, caused by the outbreak of Covid-19, has had a significant 
effect on different facets of class assessment. Teachers have had to face serious 
challenges, such as the method to be adopted, task design or handling student 
feedback, in order to ensure effective student evaluation. 


In view of the above-mentioned points, the two questions that guided the present 
study are the following. 


e What are the perceived advantages and disadvantages of using online 
assessment tools to measure students’ progress? 


¢ How do teachers perceive their experience of online student assessment 
during challenging times? 


2. Method 


The current research examined how a group of 34 teachers (21 Spaniards and 
11 Finns) teaching different online language courses (English, French, Finnish, 
Russian, Spanish, and Swedish) in primary, secondary, university, and private 
language school education used different digital assessment tools to assess their 
students online, and what their attitudes toward online assessment were. To select 
our participants in both countries, an email explaining the goal of our study was 
sent to different teachers in the different educational sectors mentioned above. The 
respondents were the cohort of the study. They were also asked to comment on the 
advantages and disadvantages of using online tools for this purpose. 


A qualitative approach was adopted, and the participants were requested to 
complete a questionnaire, consisting of 19 questions, in order to appraise their 
views on (1) the different online assessment tools they use, (2) the advantages and 
disadvantages of online assessment, and (3) their experience of online assessment 
during the pandemic. The data collated were classified in order to match the two 
research questions. 
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3. Results and discussion 


While 73.5% of the teachers were already using online assessment tools before 
Covid-19, around 40% had five or more years of experience with them, and 41.2% 
of the teachers self-evaluated themselves as experts, leaders, or pioneers (B2 to C2) 
in digital competence. 


When asked what online tools they used for assessment, there were in total more 
than 60 different online tools that those teachers used for assessment. The most 
common ones were Kahoot, Moodle, Quizlet, Google Forms, Socrative, Flipgrid, 
and Mentimeter. These were used for various purposes; 91.2% of teachers 
employed them to assess writing, 82.4% for reading, 70.6% for speaking, and 
58.8% for listening. 


When the participants were asked an open-ended question about the advantages of 
using online assessment tools, many of them mentioned that they provide instant 
feedback for both students and teachers. They also reported that these online tools 
tend to be faster and that they allow for easier correction, a finding emergent in 
another study (Boitshwarelo et al., 2017). Therefore, teachers could have more 
free time to plan other constructive activities for their students. Furthermore, 
such online tools can permit teachers and students to work at any time and in any 
place, thereby helping students to feel more motivated and engaged in their tasks, 
since they can choose when to perform them. Progress can be monitored faster as 
instantaneous feedback is provided online in most cases, which can be useful for 
identifying students’ strengths and weaknesses. 


Our participants also mentioned that, once created, quizzes and similar tasks can 
be exploited ceaselessly among students of different groups without incurring 
any major extra work; also, it is easy to recycle materials year after year. One 
noteworthy observation was that the efficiency of such activities emanates from 
the ability to evaluate a wide range of topics in one test, compared with assessment 
where responses need to be constructed (Brady, 2005). Another benefit pointed out 
is the fact that no hard copies are needed, since task completion takes place online. 
This is a cheaper option for students because of not having to purchase different 
materials. Additionally, automatic correction was sometimes used by some of the 
teachers in their assessment of student task performance. 


It was also argued that some applications like Socrative correct students’ answers to 
their quizzes automatically and compile useful statistics about their performance, 
which saves teachers a great deal of time while also providing them with important 
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information, such as their students’ progress in grammatical or lexical resources. 
Consequently, this feedback can be helpful for future task preparation. 


In terms of the disadvantages of using online assessment tools to measure students’ 
progress, the factor of trust emerged more than once. Many teachers felt that it is 
difficult to control what tools students might be using as a support for their online 
language tests and evaluation. As most of the apps employed by the participants 
do not seem to be convenient for providing feedback, online quiz tools such as 
Socrative might be more appropriate to this end. Some stated that the main drawback 
of almost all the applications available is that there is no guarantee that the person 
logging in to do the evaluation is, in fact, their student; as their cameras are turned 
off in certain cases, there exists the suggestion that cheating can be encouraged. 
Perhaps teachers should make their own choice regarding the online tools for their 
students in order that effective learning takes place; similarly, it might be advisable 
for students to have their cameras turned on while being assessed online. 


Our participants also reported that some online tools do not allow for flexibility 
in assessing, for example, recognizing partially correct answers; moreover, 
creating such grading schemes would be technically very complex. In addition, 
they considered that designing tasks that use online assessment tools is often time- 
consuming, in terms of planning, creating, checking, testing, correcting, retesting, 
and so on. 


Last but not least, one disadvantage relating to the physical state was highlighted. 
According to some teachers, working long hours online on a daily basis can put 
strain on the body, particularly the eyes. 


4. Conclusions 


This study provides novel insights into how teachers perceive the use of online 
assessment tools to monitor their students’ progress. It has been noted that there is a 
wide range of tools that have proved to be useful in assessing students’ progress in 
the receptive and productive skills, that is, reading, listening, writing, and speaking. 
One important point is that online assessment tools permit teachers and students 
to work where and when they choose. In other words, online assessment tools 
foster flexibility in learning, and teachers are afforded more free time to devote 
to planning activities that will be conducive for effective learning to take place. 
In some teachers’ views, the best option would be a combination of online and 
face-to-face assessment for reliability and credibility. However, due to its small 
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scale, the findings of this study cannot be generalized, and more research should be 
conducted in this area to enhance both the teaching and learning activity. 
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Abstract. This paper explores the use of a Computer Assisted Language Learning 
(CALL), growth mindset infused, student partnership approach to language 
learning in a primary school context. Early success is important in language 
learning. Students may doubt their ability to learn a language, especially if they 
have had a non-positive experience of learning a language. This research study 
looks at a research intervention for learning Spanish in an Irish primary school. 
A combination of in-class and CALL resources were used to teach Spanish in six 
sessions to 120 students from six different classes (10-12 years of age), all of whom 
were beginners. Most had learnt Irish since four or five years of age and had no 
previous exposure to CALL, growth mindset, or a student partnership. There were 
several logistical hiccups along the way, including a low response rate to the end of 
intervention survey (via questionnaire). However, the overall indications are that the 
students enjoyed the approach and that it could be used in other contexts. 


Keywords: CALL, growth mindset, student partnership, primary school. 


1. Introduction 


A growth mindset is helpful for learning (Dweck, 2017). CALL, learner autonomy 
(Little, Dam, & Legenhausen, 2017) and a student partnership approach (Cook- 
Sather, 2014) are also helpful approaches to the language learning process. In 
Ireland, Irish is acompulsory subject for most students and is generally not a popular 
subject (Darmody & Daly, 2015) and this can impact negatively on future language 
activities. The motivation behind this research was to see if learning Spanish with 
CALL resources and a student partnership and growth mindset approach would 
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be logistically feasible in an Irish primary school. A growth mindset approach 
might have the potential to mitigate some of the negative experiences of learning 
Irish. This paper reports on a six-week CALL-infused Spanish language course 
for primary school students (n=120, 10-12 years of age) in an English-medium 
primary school in Ireland. There were several logistical hiccups along the way, but 
the feedback from students and teachers (via questionnaires) was positive. This 
multi-pronged approach provides a template for enjoyable language teaching and 
learning in a primary school context. This approach could also be used for other 
languages, particularly less commonly taught languages including Irish (Ward, 
2015). 


Growth mindset is a belief that abilities can be developed through dedication and 
hard work (Dweck, 2017). It contrasts with a fixed mindset which is a belief that 
intelligence and talent are innate and unchangeable. If a learner has had a non- 
positive experience of learning one (L2) language, this could negatively impact on 
their learning of another language. Learners could start to think that they are not 
good at learning languages. 


Successful language learning components include engagement, motivation 
(Dornyei & Ushioda, 2013), early success, learner autonomy, and an active 
learning approach. Early success (Guskey, 2021) is sometimes overlooked, but it 
can be a key element in ensuring that learners continue to persevere when things 
get difficult. A student partnership approach, particularly giving students choice 
and autonomy in their learning, is becoming an area of focus at higher education 
but less so at primary and post-primary levels to date. 


In Ireland, students do not learn a Modern Foreign Language (MFL, e.g. French, 
German, or Spanish) until post-primary school but they learn Irish for 13 years. 
Their experience is often not very positive but previous research has shown that 
students like to use CALL resources for Irish (e.g. Purgina, Mozgovoy, & Ward, 
2017; Ward, 2007). The motivation behind this research was to see if a growth 
mindset could help to foster a positive disposition towards language learning. 


RQ1. Would it be logistically feasible to deliver a short CALL-infused 
language course in an Irish primary school? 


RQ2. Would the students have a growth mindset after doing the course? 


RQ3. Would the students enjoy/not enjoy the approach, including an 
element of choice on a learning topic)? 
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RQ4. Could the positive experience with this approach overcome the 
negative experience of learning Irish? 


2. Method 


In order to test the viability and possible effectiveness of a CALL-infused, growth 
mindset approach with primary school learners, a tailored Spanish language course 
was developed. The course was a six-week course (30 minutes each week) and 
was delivered to six different classes (10-12 years of age, with approximately 
20 students in each class). There was a mix of in-class sessions, slides, videos, and 
quizzes. The students were encouraged to share their own linguistic knowledge 
from other (home) languages (e.g. Polish or Romanian). The information was 
conveyed to the students in age-appropriate language. The students were told 
that ‘everyone can learn a language’, ‘it’s OK to make mistakes’, and “Learning a 
language is fun’ at the start of the six-week sessions. The motivation behind this 
is that some students think they are not good at languages and ‘cannot’ learn one. 
They are afraid of making mistakes as their errors (in Irish) are usually highlighted 
and corrected. They do not consider learning a language (Irish) as a fun activity. 
They would not have been told these growth mindset messages directly before. 
The students were encouraged to guess what new words meant and to ask as many 
questions as they liked. 


3. Results and discussion 


There were several logistical challenges during this research intervention (e.g. 
teacher illness and other school events). Due to related logistical reasons, the paper- 
based feedback questionnaire for the students was replaced by a shortened online 
survey (with only four closed questions and one open question) and the response 
rate was low (n=32) (much lower than an in-class paper-based questionnaire). Note 
that the questionnaire used age-appropriate questions. The in-class feedback and 
enthusiasm from the students and teachers were very positive, but several factors 
need to be considered. These include the Hawthorne effect and the novelty effect 
(a new subject, a new ‘teacher’, and a break from normal classroom activities). The 
results presented here (Table 1) should be viewed with this in mind. The majority 
of the students enjoyed learning Spanish (87.5%) and they enjoyed the approach 
(83.9%). Just over half thought they learnt some Spanish (56.3%) and would like 
to use the same approach to learning Irish (56.2%). The quiz was the most liked 
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CALL resource. The vast majority agreed that ‘anyone can learn a language’ (94%) 
and ‘it is OK to make mistakes’ (97%). A majority (63%) agreed that ‘learning 
a language is fun’. Five of the six teachers responded and they agreed that the 
students enjoyed the approach and that it would be good for teaching Irish. 


Table 1. Summary of student feedback 


Question No Alittle | Yes 

Did you enjoy learning Spanish? 0% 12.5% | 87.5% 
Did you learn any Spanish? 0% 43.7% | 56.3% 
Did you enjoy the approach to learning Spanish in class? | 0% 16.1% | 83.9% 
Would you like the same approach to learning Irish? 21.9% (21.9% | 56.2% 


Although this was a relatively short intervention, there are several valuable 
insights to be gleaned. It was logically logistically feasible to deliver a short 
CALL-infused Spanish course in an Irish primary school (RQ1), although 
flexibility and adaptability are required. One defect of this intervention is that 
there was no pre-survey administered to the students in relation to their attitudes 
towards a growth mindset in the context of language learning. However, the 
responses from the students indicate that they have taken on board the three main 
aspects of the growth mindset as outlined to them (it was not explicitly named 
for them) and they reported a positive attitude towards language learning (RQ2). 
Even taking all the caveats mentioned above, it was obvious that the students 
enjoyed the approach (RQ3). They were very engaged during the sessions and 
the positive experience appears to have offset some of the negative attitudes 
towards learning Irish (RQ4). 


Even though all six classes were in the same school, each class was different 
(different questions, engagement levels, teacher interaction). There is often an 
emphasis in some CALL and second-language acquisition research on the use of 
pre- and post-evaluations of students after a particular intervention. While in some 
educational contexts this is possible, due to the individual differences in school 
classes, it can be more difficult in a primary school setting as there are too many 
different variables. 


In terms of limitations, it should also be noted that the students have a generally 
positive attitude towards Spanish, even before starting to learn the language 
as it is seen as useful and ‘cool’. The intervention was short and the positive 
effects may not be sustained over a longer period of time. There would be a need 
to enhance the suite of resources for teachers and scaffold them to deliver the 
materials themselves in the future. 
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4. Conclusions 


This paper provided an overview of a CALL-infused intervention for teaching 
Spanish in an Irish primary school context. It aimed to give students a growth 
mindset to the process so that they would enjoy it and learn more. It allowed them 
an element of choice as part of the learning process. Overall, the results were 
positive and could be used as a model for other similar interventions in future. 
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Abstract. Game Based Language Learning (GBLL) is an approach to language 
learning in which learning happens in the process of playing a game. GBLL 
has several potential benefits including enhanced reading, writing, and cultural 
awareness. This paper provides a brief overview of Cipher — Faoi Gheasa, a GBLL 
app for Irish and learners’ feedback on the game. The app inverts the current 
Intelligent Computer Assisted Language Learning (ICALL) approach to error 
detection and it asks players (learners) to spot the deliberate errors (ciphers) in texts 
that have been changed by an evil character called Sypher. Many elements of game 
playing are included in the app. It has been played by over 150 primary school 
learners of Irish. While there is still room for improvement, the overall feedback 
from the students was very positive. 


Keywords: ICALL, less commonly taught languages, LCTLs, natural language 
processing, Irish. 


Ale Introduction 


Error diagnosis, feedback, and correction is one area of focus within ICALL 
research (Heift & Schulze, 2007; L’haire & Faltin, 2003). Learner corpora can 
provide valuable insights into the language learning process. Analysis of learner 
corpora can provide error statistics which can be used to create focused CALL 
resources (Granger, 2003). As is often the case, there are fewer learner corpora for 
Less Commonly Taught Languages (LCTLs). Often, teachers have to rely on their 
own experience, intuitions, and anecdotal evidence in terms of error frequency. 
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Learner corpora are not always readily available to substantiate these teacher 
intuitions. What can CALL researchers do in this situation? One solution is to 
gather digital resources to build up learner corpora in LCTLs. 


GBLL is an educational approach in which language learning activities happen in 
the context of a game and are enhanced by rich game mechanics (Liu, Wang, & 
Lee, 2021). GBLL can encourage reading and writing (Karadag, 2015) and also 
cultural awareness (Cosovié & Ramic-Brki¢, 2020). If learners cannot recognise 
errors, their learning can become fossilised. It can also inhibit their ability to learn 
the correct form of vocabulary. GBLL is a useful strategy to motivate learners to 
read (more enjoyable) texts that they would not normally read and this can help 
them to acquire new vocabulary and idioms. 


There are several challenges to teaching Irish in schools. It has a complex socio- 
cultural role in Irish society, teachers are not native speakers and from some 
perspectives it lacks utilitarian value. The orthography is not transparent and it is 
a morphologically complex language. Although there are some resources for Irish 
(e.g. Irish WordBricks, Purgina, Mozgovoy, & Ward, 2017, abair.ie), there is a 
lack of CALL resources for the language, which impacts on the potential to enjoy 
learning the language (see Ward, 2014). 


Content is critical in any language learning situation. In some CALL apps, the 
issue of cultural heritage and hinterland is not really addressed — maybe because 
it is assumed to be understood by the learners. GBLL apps can be a platform 
for incorporating cultural awareness within the game environment. This paper 
provides an overview of a GBLL app that encourages learners to notice errors 
while playing a game. 


2. Method 


Cipher — Faoi Gheasa is a GBLL app for Irish. It centres around an evil character 
(Sypher) that tries to make Irish mythology and folklore texts indecipherable by 
deliberately ‘misspelling’ words in the text. The players have to read the text, 
locate the errors (spells) and find the common features of these errors. They 
get points for errors spotted and can ask for hints (which cost points) from 
game characters from Irish mythology. If they are low on points, they can write 
a sentence in Irish that relates to the story. The game is adaptive and the text 
level and the game difficulty will change based on the player’s ability. Figure | 
shows a screenshot of the Cipher — Faoi Gheasa game. For illustration purposes, 
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the errors have been highlighted and the correct version of the word displayed 
underneath. 


Figure 1. Cipher — Faoi Gheasa screenshot 


Figure 2. Corpus collection element of Cipher 


Warning: Syfer is winning! x 


Quick, eg the story ending! Writing your own ending of this Oi 


tale in Trish below: 


on 


From a CALL perspective, there are two main aims, namely to gain insights into 
learners’ knowledge of errors through their interaction with the game and to collect 
a (small) corpus of learner texts from the players. Our GBLL approach inverts the 
traditional ICALL error detection model. Instead of the ICALL resource detecting 
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learners’ errors, the Cipher — Faoi Gheasa game gets the learner (player) to detect 
the errors. The Cipher — Faoi Gheasa pipeline uses several natural language 
processing tools to create the game. It uses (semi)automatic text pre-processing 
tools to generate game texts. Figure 2 above shows the learner corpus collection 
element of Cipher — Faoi Gheasa. The player is encouraged to write a sentence 
in Irish to finish off the story before the evil Sypher manages to delete the story 
forever. More details on the Cipher — Faoi Gheasa game are available at Xu, 
Dhonnchadha, and Ward (2022a). 


The Cipher — Faoi Gheasa was initially pilot tested in an Irish-medium primary 
school and was subsequently tested in an English-medium school. There were nine 
classes who tested the app with over 150 participants. They ranged in age from nine 
to 12 years of age. Some of the students played the app in pairs due to logistical 
reasons and likewise, they filled out an online questionnaire in pairs. There were 
64 responses in total. 


3. Results and discussion 


In terms of building a learner corpus, there were two questions: what would the 
players’ level of Irish be and would they be able to write sentences that made 
sense? Unfortunately, their level was very low and the feature to check that only 
(reasonably) grammatically correct Irish sentences were entered had to be disabled; 
39% of the texts were in Irish, but not relevant to the story, while only 16% were 
in Irish and relevant to the story. There is a need to enhance this aspect of the app 
in future versions. 


In terms of the game itself, the feedback from students was very positive. It 
must be borne in mind that the students did not often have the opportunity to use 
laptops in class and the Hawthorne effect could be in play. Nevertheless, they 
reported that the game was fun (46%) while only 12% said it was boring, 65% 
said they thought the texts were at the right level of difficulty for them, and 19% 
thought they were easy or too easy, while 16% thought they were either hard or 
too hard. Only 9% thought that learning or reading Irish through the game was 
bad or very bad, with 18% saying it was OK, 29% saying it was good, and 44% 
saying it was very good. Table | shows a summary of the students’ responses. 


The learners’ ability to write in Irish was limited. There were changes made 
to the game that would allow them to enter any sentence, regardless of its 
grammaticality. The learners will need more scaffolding in future to help them 
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write relevant, grammatical sentences in Irish. See Xu et al. (2022b) for more 
details. 


Table 1. Summary of student feedback 


What did you think Were the texts at How would you compare 
of the game? the right level? learning or reading Irish 
through the game to 


normal classroom teaching 
Fun (46%), challenging Too easy (9%), easy (10%), | Very bad (7%), bad (2%), 
(24%), easy (16%), boring | right level (65%), difficult /OK (18%), good (29%), 
(12%), other (2%) (12%), too difficult (4%) very good (44%) 


4. Conclusion 


The GBLL approach was feasible, although some points need to be borne in mind. 
The learners enjoyed playing the game and they were enthusiastic about it. They 
even discussed Irish in pairs, something which the teachers mentioned was very 
rare. The teachers were happy with the game and how their students played with it. 
There are future improvements planned and research will continue into this novel 
approach to noticing within a GBLL framework. 
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1. 


Abstract. Language researchers and teachers have long been interested in the 
timing of learning, and the distributed practice effect, whereby greater inter-session 
intervals result in longer retention, is well-known (Kim & Webb, 2022). Many L2 
studies have focused on the intentional learning of lexis (Edmonds, Gerbier, Palasis, 
& Whyte, 2021), neglecting implicit learning and syntactic development (Rogers, 
2021). The present project includes both explicit vocabulary learning activities 
and incidental exposure to a complex syntactic structure via a bespoke online L2 
English learning platform. The goal is to investigate the two types of learning in 
two spacing conditions. This paper describes (1) the learning activities created to 
present opportunities for explicit vocabulary learning and the concealed syntactic 
input, and (2) the tests used to evaluate participants’ receptive and productive 
knowledge of target items. It aims to inform computer assisted language learning 
design with respect to pedagogical progression, learning activities, feedback, and 
learning schedules. 


Keywords: second language acquisition, implicit/explicit learning, lexis, syntax, 
spacing effect, distributed practice. 


Introduction 


This paper presents a project on spacing effects in the online learning of lexis 
and syntax in second language (L2) English: ESPACE L2 (ESPacement dans 
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ESPACE L2: exploring spacing effects in explicit and implicit online learning... 


l’Apprentissage de la Complexité En L2; spacing in the learning of complexity in 
L2). The project lies at the intersection of L2 teaching and learning, linguistics, 
and cognitive science, and involves the creation of an online platform to deliver 
language learning activities and collect data on learners’ performances. 


2. The L2 spacing effect 


The spacing effect has been called “one of the most robust phenomena in 
experimental psychology” (Ellis, 1995, p. 118) and research in many domains 
has consistently demonstrated that the organisation of learning sessions over 
time (spacing) significantly influences memorisation (retention; cf Gerbier & 
Toppino, 2015). In studies which investigated item learning (e.g. an L2 word 
and its L1 translation), learners who attempted to retrieve items repeatedly in a 
single learning session (massed condition) made greater initial gains than those 
whose learning was spaced over several sessions (distributed condition), but the 
longer-term retention of the learners on a distributed schedule was consistently 
superior (distributed practice effect; cf Kim & Webb, 2022). Recent L2 studies 
have investigated uniform versus expanding schedules (Kang, Lindsey, Mozer, 
& Pashler, 2014), adaptive spacing (Lindsey, Shroyer, Pashler, & Mozer, 2014), 
ecological learning contexts (Rogers & Cheung, 2020), and implicit versus explicit 
learning (Nakata & Elgort, 2020). 


The ESPACE L2 project builds on this work by comparing two types of learning 
— explicit learning of vocabulary and implicit learning of syntax — in two different 
learning schedules, using a pre-, post-, and delayed post-test design. Participants 
are non-English majors at the Université Céte d’ Azur, randomly assigned to a 48- 
hour (sessions every two days) or seven-day learning schedule (sessions every 
week). 


3. Pedagogical design 


The platform includes nine main learning activities organised in three sessions of 
three activities, focusing on the meaning, form (sound and spelling), then use of 16 
English verbs, with immediate feedback to learners after each attempt. In addition 
to the activities shown in Table 1, the first session included an initial presentation 
activity (pronunciation, definition, translation, and example sentence for each 
verb); each session ended with an opportunity to review learning, for a total of 
13 exercises taking approximately 110 minutes. 
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Table 1. Lexical learning activities 
SESSION 1 SESSION 2 SESSION 3 
1. MEANING | synonym translation synonym 
multiple choice multiple choice gapfill 
prompt to loathe Most of us loathe The TV show received 
asking for help, so good reviews from 
we continue alone the critics, but was 
____ by the public 
answer to hate someone détester loathed 
or something 
2. FORM multiple choice anagram dictation 
prompt [audio] ATLOEH [audio] 
answer loathe LOATHE loathe 
3. USE gapfill dialogue listening dialogue listening 
multiple choice multiple choice sentence completion 
prompt [text] The UK [text] Angela is [text] Julia and 
Independence Party complaining to her Frances are discussing 
(UKIP) has always friends about her job. | the difficulties of 
campaigned for Her friends ask: campaigning for 
Britain to leave the climate change. 
European Union. [audio] What do 
Because it focused you loathe doing [audio] Julia: Of 
only on this question, | the most? course, lots of 
some commentators climate change 
wondered how UKIP _ [prompt] Her protestors absolutely 
could be so successful. | friends ask what loathe the media. 
Indeed, only the fact Frances: Sorry I didn’t 
that UKIP supporters catch that. What do 
the protestors loathe? 
continental Europe 
keeps them together. [prompt] Frances is 
wondering what 
answer loathe Angela loathes the protestors loathe 
doing the most 
4. SELF- Indicate for each as Session 1 as Session 1 
EVALUATION | of 16 verbs 
- I don’t know it yet 
- [know it a little 
- [know it well 


Table | shows how the learning programme progressed from recognition to more 
elaborate production tasks; learners’ progress through the sessions and activities 
was also controlled to prevent repetition, to record only first responses, and to keep 
the number of exposures to target items constant. 


Concerning the choice of learning targets, the verbs selected for explicit learning met 
three criteria: they belonged to the 5,000-6,000 frequency band, were characteristic 
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of learners at C2 level in the English Vocabulary Profile, and were not French 
cognates. The target sentences for implicit syntactic learning were embedded in the 
third activity of each session through the wording of the example paragraphs and 
answer prompts, and were controlled for lexical variation. 


4. ESPACE L2 research design and test battery 


Participants connected to the platform six times: (1) to enrol, provide informed 
consent, and give a linguistic biography, (2) to take five pre-tests listed in Table 2 
and schedule learning sessions, (3-5) to complete the learning sessions, including 
an immediate post-test at the end of Session 3, and (6) to take the delayed post-test. 


Table 2. Tests of proficiency, vocabulary knowledge, and syntactic development 


Test battery Pre-test | Immediate | Delayed 
post-test post-test 
1. PROFICIENCY | V_YesNo V Xx xX 
(Meara & Miralpeix, 2016) 
2. VOCABULARY | Modified Vocabulary V V V 


Knowledge Scale 


(Paribakht & Wesche, 1993) 

- I’ve never seen this verb 

- I’ve seen this verb but 

I don’t know what it means 

- I’ve seen this verb [provide 
synonym/translation/definition 
and use in sentence] 


3. READING Self-paced reading V V V 


(reaction times for target 
and spillover segments) 

4. LISTENING Aural acceptability V V V 
judgement 


(correct/incorrect scores 
on grammatical and 
ungrammatical exemplars) 


5. WRITING Sentence completion V xX V 


(production of target structure) 


The first test listed in Table 2 situates the learners in terms of general proficiency, 
while the second provides our measure of vocabulary knowledge before and after 
the intervention. Since implicit syntactic learning via 48 exposures to examples 
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of the target structure is likely to be difficult to detect, three syntax tests were 
included, ranging from the most implicit measure, self-paced reading, through an 
acceptability judgement task, to the most explicit measure, sentence completion (cf 
Vafaee, Suzuki, & Kachisnke, 2017). The project platform also collects responses 
for each participant on all learning activities. 


5. Conclusion 


The ESPACE L2 project allows for the collection of rich data for the investigation 
of spacing schedules for lexical learning and syntactic development. Results will 
contribute to our understanding of the effects of timing of practice and evaluation 
sessions in both explicit and implicit conditions of L2 learning. 
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Abstract. In recent years, increasing attention has been paid to the effects of cultural 
factors on cultural identity in the field of teaching Chinese as a foreign language. 
The central issues include the kind of knowledge structure and value orientations 
involved in Chinese teaching and Chinese textbooks, the responsibility teachers 
take when educating people, and ways of further optimizing and improving the 
quality of education. This paper reports a study exploring the cultural factors in 
a course of intercultural communication offered to Chinese language majors at 
a university in Beijing. The researchers followed the students in this course for 
four months, from September to December of 2021. The researchers analyzed 
the value orientations of the course textbook Boya Chinese and the ways of 
using the textbook in classroom teaching. The purpose was to identify the basic 
characteristics and existing problems of the textbook and its relationship with 
language teaching. Through data analysis, four dimensions of cultural identity 
were identified. It is suggested that through strengthening the blended mode of 
online and offline teaching, teachers can help students understand and appreciate 
the Chinese culture and language so as to build a community with a shared future 


for mankind. 


Keywords: cultural identity, cross-cultural communication, Boya Chinese, field 


analysis. 
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1. Introduction 


In recent years, the influence of cultural factors on cultural identity has received 
increasing attention in teaching Chinese as a second language (Sun, 2021). 
Cultural identity is a sense of group cultural identity and a sense that an individual 
is influenced by group culture. Many researchers have studied the cultural factors 
of foreign Chinese teaching materials (e.g. Chen, 2022; Du, 2021; Luo, 2022; 
Xu, 2013). There is not enough awareness of the knowledge structures and value 
orientations involved in language teaching and materials and the responsibilities 
they assume (Bai & Zhang, 2022). 


The purpose of this study is to clarify the basic characteristics of the Chinese 
textbook Boya Chinese, identify the problems, and seek possible solutions in 
relation to Chinese language teaching and learning. The research questions are as 
follows. 


(1) What are the cultural identities presented in the course textbook of 
‘Intercultural Communication’? 


(2) What are the characteristics of the students’ cultural identities? 


(3) How can the teacher integrate cultural identities presented in the 
materials through teaching? 


(4) How can the teacher integrate students’ identities through teaching? 


2. Method 


This study was conducted in a second-year core course ‘Intercultural 
Communication’ offered to the Chinese language majors at a university in 
Beijing. Data included a questionnaire survey, classroom observations, eight semi- 
structured interviews, and the textbook of the course. 


The questionnaire method is a four-month follow-up survey around online 
teaching in the fall semester of 2021-2022; 100 questionnaires were collected 
and the results were analyzed using SPSS. The interviews were conducted from 
September to December 2021 with eight international students in a one-hour 
interview each. 
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cs Data analysis and discussion 


3.1. Cultural identity revealed in the textbook 


3.1.1. Theme 


The analysis of themes is divided into political, social and cultural, historical 
and geographical, moral and quality, economic and commercial, science and 
technology, and environmental protection (Wang, 2021). There are 16 topics in the 
textbook Boya Chinese (Intermediate I, Table 1, Li, 2012). 


Table 1. Thematic education categories in Boya Chinese 


Moral Socio-cultural | Science and History and 
category (2) category (9) technology Geography 
category (3) Class (2) 
Boya Chinese | Love Rose Three E-mails | Children Sun Yat-sen 
(Intermediate) learning 
language 

Your husband | Diary Color and Wu Zetian 
is So nice. personality 

Study Abroad | What should 

in China be done 

Is she our 

daughter? 

Recordings 

Music and 

Neighborhood 

Girls 

Eat in China 

Treating guests 

to dinner 

National 

superstitions 


3.1.2. _ Character image 


The characters in the textbook present their own viewpoints (Sun, 2021). For 
example, the description of the characters of Sun Yat-sen and Wu Zetian in the 
textbook, which implies the language, behavior, and intercultural communication 
functions. Through the interactions using the characters, intercultural 
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communication problems are highlighted and their intercultural communication 
skills are cultivated. 


3.1.3. Geographical distribution 


Bourdieu and other researchers argue that all human activities take place in different 
socially constructed fields (Bourdieu & Wang, 1995). Different localization has 
different cultural capital, which leads people to compete and chase each other. They 
accumulate capital continuously, in order to maximize their own social advantages. 


The analysis shows that the textbook showed the spiritual outlook of China’s big 
cities from texts such as ‘Eat in China’ and ‘Study in China’, which changed their 
stereotypes. 


3.1.4. Cultural factors 


This study uses the database statistical model to investigate the texts with Chinese 
cultural factors in the book. Among them, the overall presentation of cultural 
factors can be divided into eight parts, accounting for more than half of the total 
number of courses (Table 2). Also, there are Chinese cultural factors in 43 classes. 


Table 2. Cultural factors in Bova Chinese 


Broad Cultural Occurrence Expression of cultural factors 
categories of Factors Number 

cultural factors | Subcategory 

Geographical City Introduction | 10 Harbin, Beijing, Xi’an, 
overview Xi’an, Xiangshan, Macau, 


Guangzhou, Wuxi, Suzhou, 
Hangzhou, Liuzhou 


Places of interest | 3 Great Wall, Forbidden 
City, Terracotta Warriors, 
Horses of Qin Shi Huang 
River 2 Yellow River, Yangtze River 
Natural Scenery | 1 Mount Tai 
Special building | 1 Temple 
Diet Special Diet 2 Sweet and Sour Pork, Tofu 
Interpersonal Appellation 5 Boss, husband, leader, elder 
sister, brother-in-law 
Modern Life Daily Life 1 Big Stall 
People Events Historical figures 5 Sun Yat-sen, Wu Zetian, 


Emperor Taizong of Tang, 
Emperor Gaozong of 
Tang, Empress Wang 
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Country Profile | Administrative | 5 Northeast, Guangdong, 
Area Shaanxi, Sichuan, County 
History 3 Qing Dynasty, Republic of 
China, Tang Dynasty 
Literary Arts Idiom 2 Exactly the same, follow 
the local customs 
Language and Language 3 Mandarin, dialect, 
Literature native language 


3.1.5. Characteristics of students’ cultural identity 


The questionnaire shows that 93.4% of the international students are satisfied with 
the distribution of the topics in the textbook, which contains Chinese traditional 
culture, 97.5% of the international students said that they had a deep understanding 
of Chinese traditional culture through the characters of Sun Yat-sen and Wu 
Zetian, and 55.7% of the international students studied the characters and went 
to the traditional Chinese cultural places involved in the characters; 92% of the 
international students visited the Forbidden City in Beijing, and 56% of them 
visited the cities mentioned in the text. 


Seven of the eight international students said in the in-depth interview that, of 
the cultural factors, they were most interested in Chinese food culture and had an 
admiration for historical celebrities. More than 80% of them said they did not know 
much about the national conditions. They also did not particularly understand the 
wording of interpersonal communication. 


3.2. Integration of the cultural identity of the textbook 
and the students’ cultural identities 


Teachers create a positive atmosphere for teaching to effectively integrate the 
cultural identity of the material with the cultural identity of the students. 


First of all, for teachers, they should constantly improve their own learning. 
Secondly, they should use teaching to enhance students’ values and students’ 
cultural safety awareness as well. 


In terms of teaching materials, we should pay attention to the integration of 
educational objectives and teaching materials, rationally compare the differences 
between the East and the West in teaching materials, and actively put forward 
suggestions for reform and innovation. 
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4. Conclusions 


In this study, the researchers explored cultural identity and identification in the 
course textbook of ‘Intercultural Communication’. It is suggested that the role of 
culture be given full play to promote the acquisition of Chinese skills, the learning 
of verbal knowledge, and the strengthening of thinking and political awareness to 
a high degree of intermingling, so that international students can become narrators 
of Chinese stories, enhance their personality to integrate ideological political 
education into education. 


The inspiration for teaching is, firstly, to improve the educational objectives; 
secondly, to pay attention to the integration of text selection and educational 
objectives; thirdly, to improve teachers’ abilities to educate people and consciously 
strengthen the leadership and cultivation of students’ values; and fourthly, to 
improve international students’ abilities to analyze, identify, and criticize foreign 
Chinese textbooks independently. 
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The 2022 EUROCALL conference was held in Reykjavik on 17-19 August 2022 as 
a fully online event hosted by the Vigdis Finnbogadéttir Institute for Foreign 
Languages, the University of Iceland, and the Arni Magnisson Institute for 
Icelandic Studies. The conference theme was Intelligent CALL, granular systems 
and learner data. 

This theme reflects the newest developments in the field of technology for 
language learning. Subfields such as natural language processing and machine 
learning not only enable smoother spoken and written communication between 
human learners and computers, but also offer ways in which language learning 
can be tailored to the needs of individual learners. By adding components of 
automatic speech recognition, text-to-speech systems, automatic feedback 
mechanisms, and tracking systems monitoring learners’ progress and their use of 
tools, applications are becoming better targeted. All of this is used to optimise the 
learning experience of individual learners. 

This volume includes 66 short papers by some of the EUROCALL 2022 presenters 


and it offers a combination of research studies and theoretical papers reflecting 


the subthemes of the conference. The articles are ordered alphabetically. 
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